Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
This is a network apply the following layout:
LayerTests.java:203 executed in 0.07 seconds (0.000 gc):
return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
.render(Format.PNG).toImage();
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine7843854477911190902/dotfile.dot -ooutfile.svg]
Returns
Using Seed 7272808493498678272
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.08, 0.7, -0.128 ] ]
]
[
[ [ -0.128, 0.7, 0.08 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.35 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11406307342986
Reset training subject: 11406336423591
Constructing line search parameters: GD
th(0)=2.1934415203839994;dx=-7.357441330695156
Armijo: th(2.154434690031884)=45.7169963241023; dx=13.451805917198378 evalInputDelta=-43.5235548037183
Armijo: th(1.077217345015942)=13.61631041443225; dx=3.047182293251612 evalInputDelta=-11.422868894048252
Armijo: th(0.3590724483386473)=3.7035292611087707; dx=-3.8892334560462327 evalInputDelta=-1.5100877407247713
Armijo: th(0.08976811208466183)=2.3555695283191316; dx=-6.490389362032926 evalInputDelta=-0.16212800793513216
Armijo: th(0.017953622416932366)=2.214379445851236; dx=-7.18403093696271 evalInputDelta=-0.020937925467236607
Armijo: th(0.002992270402822061)=2.196532296985491; dx=-7.328539598406415 evalInputDelta=-0.003090776601491374
Armijo: th(4.2746720040315154E-4)=2.193873291466424; dx=-7.353312511796766 evalInputDelta=-4.3177108242442586E-4
Armijo: th(5.343340005039394E-5)=2.193495313698926; dx=-7.356925228332857 evalInputDelta=-5.379331492649797E-5
Armijo: th(5.9370444500437714E-6)=2.193447494906535; dx=-7.357383985988234 evalInputDelta=-5.97452253536801E-6
Armijo: th(5.937044450043771E-7)=2.1934421178079884; dx=-7.357435596224464 evalInputDelta=-5.97423988946133E-7
Armijo: th(5.397313136403428E-8)=2.1934415746950116; dx=-7.357440809379639 evalInputDelta=-5.43110121320467E-8
Armijo: th(4.4977609470028565E-9)=2.193441524909915; dx=-7.357441287252197 evalInputDelta=-4.52591564226168E-9
Armijo: th(3.4598161130791205E-10)=2.1934415207321467; dx=-7.35744132735339 evalInputDelta=-3.481472887756354E-10
Armijo: th(2.4712972236279432E-11)=2.193441520408867; dx=-7.357441330456458 evalInputDelta=-2.4867663483973956E-11
Armijo: th(1.6475314824186289E-12)=2.1934415203856577; dx=-7.357441330679243 evalInputDelta=-1.6582291095801338E-12
Armijo: th(1.029707176511643E-13)=2.193441520384103; dx=-7.3574413306941615 evalInputDelta=-1.0347278589506459E-13
Armijo: th(6.057101038303783E-15)=2.1934415203840056; dx=-7.357441330695098 evalInputDelta=-6.217248937900877E-15
MIN ALPHA (3.3650561323909904E-16): th(0.0)=2.1934415203839994
Fitness changed from 2.1934415203839994 to 2.1934415203839994
Static Iteration Total: 0.3366; Orientation: 0.0038; Line Search: 0.2761
Iteration 1 failed. Error: 2.1934415203839994
Previous Error: 0.0 -> 2.1934415203839994
Optimization terminated 1
Final threshold in iteration 1: 2.1934415203839994 (> 0.0) after 0.337s (< 30.000s)
Returns
2.1934415203839994
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.7, 0.08, -0.128 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.01 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ -0.576176, 1.005728, 1.024032, 0.8050400000000001, -0.500656, -0.510384 ] ]
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.73 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11406667063377
Reset training subject: 11406679765644
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=2.1934415203839994}, derivative=-7.357441330695156}
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=2.193441520484626}, derivative=-7.3574413297292764}, evalInputDelta = 1.0062661814913554E-10
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=2.1934415210883813}, derivative=-7.357441323933999}, evalInputDelta = 7.043818861518503E-10
F(5.384615384615385E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204381827}, derivative=-7.357441330175067}, evalInputDelta = 5.418332449380614E-11
F(3.7692307692307697E-10) = LineSearchPoint{point=PointSample{avg=2.193441520763282}, derivative=-7.357441327054533}, evalInputDelta = 3.792823832782233E-10
F(2.899408284023669E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204131748}, derivative=-7.357441330415108}, evalInputDelta = 2.9175328819519564E-11
F(2.0295857988165684E-10) = LineSearchPoint{point=PointSample{avg=2.1934415205882285}, derivative=-7.35744132873482}, evalInputDelta = 2.0422907809347635E-10
F(1.561219845243514E-11) = LineSearchPoint{point=PointSample{avg=2.19344152039971}, derivative=-7.357441330544362}, evalInputDelta = 1.5710543976865665E-11
F(1.0928538916704598E-10) = LineSearchPoint{point=PointSample{avg=2.1934415204939692}, derivative=-7.357441329639592}, evalInputDelta = 1.09969811035171E-10
F(8.406568397465076E-12) = LineSearchPoint{point=PointSample{avg=2.193441520392459}, derivative=-7.357441330613959}, evalInputDelta = 8.459455358433843E-12
F(5.884597878225553E-11) = LineSearchPoint{point=PointSample{avg=2.193441520443214}, derivative=-7.3574413301267745}, evalInputDelta = 5.92144111521975E-11
F(4.119218514757887E-10) = LineSearchPoint{point=PointSample{avg=2.1934415207985}, derivative=-7.357441326716487}, evalInputDelta = 4.1450043397617264E-10
F(3.168629626736836E-11) = LineSearchPoint{point=PointSample{avg=2.193441520415884}, derivative=-7.357441330389106}, evalInputDelta = 3.1884717088814796E-11
F(2.2180407387157852E-10) = LineSearchPoint{point=PointSample{avg=2.193441520607192}, derivative=-7.357441328552796}, evalInputDelta = 2.2319257553249372E-10
F(1.706185183627527E-11) = LineSearchPoint{point=PointSample{avg=2.193441520401168}, derivative=-7.35744133053036}, evalInputDelta = 1.716848885280342E-11
F(1.194329628539269E-10) = LineSearchPoint{point=PointSample{avg=2.1934415205041797}, derivative=-7.357441329541578}, evalInputDelta = 1.2018031014804365E-10
F(9.187150988763607E-12) = LineSearchPoint{point=PointSample{avg=2.1934415203932445}, derivative=-7.35744133060642}, evalInputDelta = 9.245049170658604E-12
F(6.431005692134525E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204487125}, derivative=-7.357441330073998}, evalInputDelta = 6.471312374856097E-11
F(4.501703984494168E-10) = LineSearchPoint{point=PointSample{avg=2.1934415208369877}, derivative=-7.357441326347052}, evalInputDelta = 4.52988313526248E-10
F(3.4628492188416676E-11) = LineSearchPoint{point=PointSample{avg=2.193441520418845}, derivative=-7.357441330360687}, evalInputDelta = 3.484545985088516E-11
F(2.423994453189167E-10) = LineSearchPoint{point=PointSample{avg=2.1934415206279163}, derivative=-7.357441328353869}, evalInputDelta = 2.439168866885666E-10
F(1.864611117837821E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204027626}, derivative=-7.357441330515057}, evalInputDelta = 1.8763213205374996E-11
F(1.3052277824864746E-10) = LineSearchPoint{point=PointSample{avg=2.19344152051534}, derivative=-7.357441329434463}, evalInputDelta = 1.3134071608078557E-10
F(1.004021371143442E-11) = LineSearchPoint{point=PointSample{avg=2.193441520394103}, derivative=-7.3574413305981805}, evalInputDelta = 1.0103473613298775E-11
F(7.028149598004095E-11) = LineSearchPoint{point=PointSample{avg=2.193441520454721}, derivative=-7.3574413300163215}, evalInputDelta = 7.072165075783232E-11
F(4.919704718602866E-10) = LineSearchPoint{point=PointSample{avg=2.1934415208790496}, derivative=-7.357441325943313}, evalInputDelta = 4.950502230371967E-10
F(3.784388245079128E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204220805}, derivative=-7.35744133032963}, evalInputDelta = 3.808109383385272E-11
F(2.64907177155539E-10) = LineSearchPoint{point=PointSample{avg=2.193441520650565}, derivative=-7.357441328136471}, evalInputDelta = 2.665654363909198E-10
F(2.037747516581069E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204045052}, derivative=-7.357441330498334}, evalInputDelta = 2.050581926482664E-11
F(1.4264232616067484E-10) = LineSearchPoint{point=PointSample{avg=2.193441520527535}, derivative=-7.357441329317403}, evalInputDelta = 1.435354057832683E-10
F(1.0972486627744219E-11) = LineSearchPoint{point=PointSample{avg=2.193441520395041}, derivative=-7.357441330589175}, evalInputDelta = 1.1041390024502107E-11
F(7.680740639420954E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204612874}, derivative=-7.357441329953288}, evalInputDelta = 7.728795381467535E-11
F(5.376518447594668E-10) = LineSearchPoint{point=PointSample{avg=2.193441520925017}, derivative=-7.357441325502087}, evalInputDelta = 5.410174530595668E-10
F(4.135783421226668E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204256164}, derivative=-7.357441330295689}, evalInputDelta = 4.161693212267892E-11
F(2.8950483948586675E-10) = LineSearchPoint{point=PointSample{avg=2.1934415206753166}, derivative=-7.357441327898888}, evalInputDelta = 2.913171925911229E-10
F(2.2269603037374367E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204064086}, derivative=-7.357441330480058}, evalInputDelta = 2.240918561824401E-11
F(1.5588722126162056E-10) = LineSearchPoint{point=PointSample{avg=2.193441520540863}, derivative=-7.357441329189473}, evalInputDelta = 1.5686341114928837E-10
F(1.1991324712432351E-11) = LineSearchPoint{point=PointSample{avg=2.193441520396066}, derivative=-7.357441330579334}, evalInputDelta = 1.2066792010045901E-11
F(8.393927298702646E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204684643}, derivative=-7.357441329884404}, evalInputDelta = 8.446487953506221E-11
F(5.875749109091853E-10) = LineSearchPoint{point=PointSample{avg=2.1934415209752527}, derivative=-7.35744132501989}, evalInputDelta = 5.912532685670158E-10
F(4.5198070069937326E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204294804}, derivative=-7.357441330258597}, evalInputDelta = 4.548095233758431E-11
F(3.1638649048956127E-10) = LineSearchPoint{point=PointSample{avg=2.1934415207023665}, derivative=-7.357441327639243}, evalInputDelta = 3.1836711045230004E-10
F(2.4337422345350867E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204084896}, derivative=-7.357441330460087}, evalInputDelta = 2.4490187655601403E-11
F(1.7036195641745607E-10) = LineSearchPoint{point=PointSample{avg=2.193441520555428}, derivative=-7.3574413290496645}, evalInputDelta = 1.7142864905395072E-10
F(1.3104765878265852E-11) = LineSearchPoint{point=PointSample{avg=2.193441520397186}, derivative=-7.35744133056858}, evalInputDelta = 1.318678499728776E-11
F(9.173336114786096E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204763074}, derivative=-7.357441329809122}, evalInputDelta = 9.230793907022417E-11
F(6.421335280350267E-10) = LineSearchPoint{point=PointSample{avg=2.1934415210301528}, derivative=-7.357441324492919}, evalInputDelta = 6.461533530455199E-10
F(4.939488677192513E-11) = LineSearchPoint{point=PointSample{avg=2.193441520433703}, derivative=-7.35744133021806}, evalInputDelta = 4.970379663404856E-11
F(3.457642074034759E-10) = LineSearchPoint{point=PointSample{avg=2.193441520731928}, derivative=-7.35744132735549}, evalInputDelta = 3.4792879688438916E-10
F(2.65972467233443E-11) = LineSearchPoint{point=PointSample{avg=2.1934415204107633}, derivative=-7.357441330438259}, evalInputDelta = 2.6763924410033724E-11
F(1.861807270634101E-10) = LineSearchPoint{point=PointSample{avg=2.193441520571346}, derivative=-7.357441328896874}, evalInputDelta = 1.8734658269181637E-10
F(1.4321594389493084E-11) = LineSearchPoint{point=PointSample{avg=2.193441520398411}, derivative=-7.357441330556828}, evalInputDelta = 1.4411583038054232E-11
F(1.0025116072645159E-10) = LineSearchPoint{point=PointSample{avg=2.193441520484878}, derivative=-7.35744132972685}, evalInputDelta = 1.0087841673112052E-10
F(7.711627748188584E-12) = LineSearchPoint{point=PointSample{avg=2.1934415203917594}, derivative=-7.357441330620671}, evalInputDelta = 7.760014852919994E-12
Loops = 52
F(0.7617320938905214) = LineSearchPoint{point=PointSample{avg=8.129727325600129}, derivative=-2.5693190297992885E-5}, evalInputDelta = 5.93628580521613
8.129727325600129 > 2.1934415203917594
Fitness changed from 2.1934415203839994 to 2.1934415203839994
Static Iteration Total: 0.7249; Orientation: 0.0005; Line Search: 0.6863
Iteration 1 failed. Error: 2.1934415203839994
Previous Error: 0.0 -> 2.1934415203839994
Optimization terminated 1
Final threshold in iteration 1: 2.1934415203839994 (> 0.0) after 0.725s (< 30.000s)
Returns
2.1934415203839994
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.7, 0.08, -0.128 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ -0.576176, 1.005728, 1.024032, 0.8050400000000001, -0.500656, -0.510384 ] ]
]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.27 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11407409646514
Reset training subject: 11407419364034
Adding measurement 708bfe77 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 2.1934415203839994 < 2.1934415203839994. Total: 1
th(0)=2.1934415203839994;dx=-7.357441330695156
Non-optimal measurement 45.7169963241023 < 2.1934415203839994. Total: 1
Armijo: th(2.154434690031884)=45.7169963241023; dx=13.451805917198378 evalInputDelta=-43.5235548037183
Non-optimal measurement 13.61631041443225 < 2.1934415203839994. Total: 1
Armijo: th(1.077217345015942)=13.61631041443225; dx=3.047182293251612 evalInputDelta=-11.422868894048252
Non-optimal measurement 3.7035292611087707 < 2.1934415203839994. Total: 1
Armijo: th(0.3590724483386473)=3.7035292611087707; dx=-3.8892334560462323 evalInputDelta=-1.5100877407247713
Non-optimal measurement 2.3555695283191316 < 2.1934415203839994. Total: 1
Armijo: th(0.08976811208466183)=2.3555695283191316; dx=-6.490389362032926 evalInputDelta=-0.16212800793513216
Non-optimal measurement 2.214379445851236 < 2.1934415203839994. Total: 1
Armijo: th(0.017953622416932366)=2.214379445851236; dx=-7.18403093696271 evalInputDelta=-0.020937925467236607
Non-optimal measurement 2.196532296985491 < 2.1934415203839994. Total: 1
Armijo: th(0.002992270402822061)=2.196532296985491; dx=-7.328539598406415 evalInputDelta=-0.003090776601491374
Non-optimal measurement 2.193873291466424 < 2.1934415203839994. Total: 1
Armijo: th(4.2746720040315154E-4)=2.193873291466424; dx=-7.353312511796766 evalInputDelta=-4.3177108242442586E-4
Non-optimal measurement 2.193495313698926 < 2.1934415203839994. Total: 1
Armijo: th(5.343340005039394E-5)=2.193495313698926; dx=-7.356925228332857 evalInputDelta=-5.379331492649797E-5
Non-optimal measurement 2.193447494906535 < 2.1934415203839994. Total: 1
Armijo: th(5.9370444500437714E-6)=2.193447494906535; dx=-7.357383985988234 evalInputDelta=-5.97452253536801E-6
Non-optimal measurement 2.1934421178079884 < 2.1934415203839994. Total: 1
Armijo: th(5.937044450043771E-7)=2.1934421178079884; dx=-7.357435596224464 evalInputDelta=-5.97423988946133E-7
Non-optimal measurement 2.1934415746950116 < 2.1934415203839994. Total: 1
Armijo: th(5.397313136403428E-8)=2.1934415746950116; dx=-7.357440809379639 evalInputDelta=-5.43110121320467E-8
Non-optimal measurement 2.193441524909915 < 2.1934415203839994. Total: 1
Armijo: th(4.4977609470028565E-9)=2.193441524909915; dx=-7.357441287252197 evalInputDelta=-4.52591564226168E-9
Non-optimal measurement 2.1934415207321467 < 2.1934415203839994. Total: 1
Armijo: th(3.4598161130791205E-10)=2.1934415207321467; dx=-7.35744132735339 evalInputDelta=-3.481472887756354E-10
Non-optimal measurement 2.193441520408867 < 2.1934415203839994. Total: 1
Armijo: th(2.4712972236279432E-11)=2.193441520408867; dx=-7.357441330456458 evalInputDelta=-2.4867663483973956E-11
Non-optimal measurement 2.1934415203856577 < 2.1934415203839994. Total: 1
Armijo: th(1.6475314824186289E-12)=2.1934415203856577; dx=-7.357441330679243 evalInputDelta=-1.6582291095801338E-12
Non-optimal measurement 2.193441520384103 < 2.1934415203839994. Total: 1
Armijo: th(1.029707176511643E-13)=2.193441520384103; dx=-7.3574413306941615 evalInputDelta=-1.0347278589506459E-13
Non-optimal measurement 2.1934415203840056 < 2.1934415203839994. Total: 1
Armijo: th(6.057101038303783E-15)=2.1934415203840056; dx=-7.357441330695098 evalInputDelta=-6.217248937900877E-15
Non-optimal measurement 2.1934415203839994 < 2.1934415203839994. Total: 1
MIN ALPHA (3.3650561323909904E-16): th(0.0)=2.1934415203839994
Fitness changed from 2.1934415203839994 to 2.1934415203839994
Static Iteration Total: 0.2646; Orientation: 0.0035; Line Search: 0.2287
Iteration 1 failed. Error: 2.1934415203839994
Previous Error: 0.0 -> 2.1934415203839994
Optimization terminated 1
Final threshold in iteration 1: 2.1934415203839994 (> 0.0) after 0.265s (< 30.000s)
Returns
2.1934415203839994
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.7, 0.08, -0.128 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.01 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ -0.576176, 1.005728, 1.024032, 0.8050400000000001, -0.500656, -0.510384 ] ]
]
TrainingTester.java:432 executed in 0.00 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
No Data
TrainingTester.java:435 executed in 0.00 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
No Data
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:370 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0042 = network_target.state();
assert temp_18_0042 != null;
String temp_18_0041 = temp_18_0042.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0042.freeRef();
return temp_18_0041;
Returns
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 1.78 seconds (0.141 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11407710698321
Reset training subject: 11407720653904
Constructing line search parameters: GD
th(0)=0.5923240879573332;dx=-0.11242199336048941
New Minimum: 0.5923240879573332 > 0.37691385449489573
END: th(2.154434690031884)=0.37691385449489573; dx=-0.08754715349654488 evalInputDelta=0.21541023346243743
Fitness changed from 0.5923240879573332 to 0.37691385449489573
Iteration 1 complete. Error: 0.37691385449489573 Total: 0.0561; Orientation: 0.0007; Line Search: 0.0259
th(0)=0.37691385449489573;dx=-0.06845397026813493
New Minimum: 0.37691385449489573 > 0.13264409262698712
END: th(4.641588833612779)=0.13264409262698712; dx=-0.03679867947067503 evalInputDelta=0.2442697618679086
Fitness changed from 0.37691385449489573 to 0.13264409262698712
Iteration 2 complete. Error: 0.13264409262698712 Total: 0.0252; Orientation: 0.0004; Line Search: 0.0162
th(0)=0.13264409262698712;dx=-0.02070918776217733
New Minimum: 0.13264409262698712 > 0.016658739614842474
END: th(10.000000000000002)=0.016658739614842474; dx=-0.0024878828402515934 evalInputDelta=0.11598535301214465
Fitness changed from 0.13264409262698712 to 0.016658739614842474
Iteration 3 complete. Error: 0.016658739614842474 Total: 0.0322; Orientation: 0.0006; Line Search: 0.0232
th(0)=0.016658739614842474;dx=-0.0020034047732510963
New Minimum: 0.016658739614842474 > 0.0029055153861502984
WOLF (strong): th(21.544346900318843)=0.0029055153861502984; dx=7.266685331296899E-4 evalInputDelta=0.013753224228692176
New Minimum: 0.0029055153861502984 > 0.002429921703500375
END: th(10.772173450159421)=0.002429921703500375; dx=-6.383681200607035E-4 evalInputDelta=0.0142288179113421
Fitness changed from 0.016658739614842474 to 0.002429921703500375
Iteration 4 complete. Error: 0.002429921703500375 Total: 0.0391; Orientation: 0.0005; Line Search: 0.0308
th(0)=0.002429921703500375;dx=-2.8153468095580157E-4
New Minimum: 0.002429921703500375 > 3.817075835563029E-4
WOLF (strong): th(23.2079441680639)=3.817075835563029E-4; dx=1.0502493885097621E-4 evalInputDelta=0.002048214119944072
New Minimum: 3.817075835563029E-4 > 2.844078842653764E-4
END: th(11.60397208403195)=2.844078842653764E-4; dx=-8.825487105241288E-5 evalInputDelta=0.0021455138192349983
Fitness changed from 0.002429921703500375 to 2.844078842653764E-4
Iteration 5 complete. Error: 2.844078842653764E-4 Total: 0.0447; Orientation: 0.0005; Line Search: 0.0365
th(0)=2.844078842653764E-4;dx=-3.2885330336707554E-5
New Minimum: 2.844078842653764E-4 > 6.79426619810632E-5
WOLF (strong): th(25.000000000000007)=6.79426619810632E-5; dx=1.556811255396255E-5 evalInputDelta=2.1646522228431318E-4
New Minimum: 6.79426619810632E-5 > 2.4758264089874862E-5
END: th(12.500000000000004)=2.4758264089874862E-5; dx=-8.658608891372419E-6 evalInputDelta=2.5964962017550154E-4
Fitness changed from 2.844078842653764E-4 to 2.4758264089874862E-5
Iteration 6 complete. Error: 2.4758264089874862E-5 Total: 0.0397; Orientation: 0.0006; Line Search: 0.0281
th(0)=2.4758264089874862E-5;dx=-2.925766388441296E-6
New Minimum: 2.4758264089874862E-5 > 1.1054579457325479E-5
WOLF (strong): th(26.930433625398557)=1.1054579457325479E-5; dx=1.9080564753246162E-6 evalInputDelta=1.3703684632549384E-5
New Minimum: 1.1054579457325479E-5 > 1.6343035499024183E-6
END: th(13.465216812699278)=1.6343035499024183E-6; dx=-5.088549565583399E-7 evalInputDelta=2.3123960539972443E-5
Fitness changed from 2.4758264089874862E-5 to 1.6343035499024183E-6
Iteration 7 complete. Error: 1.6343035499024183E-6 Total: 0.0407; Orientation: 0.0004; Line Search: 0.0330
th(0)=1.6343035499024183E-6;dx=-2.200107243643088E-7
Armijo: th(29.00993021007988)=2.170778511052444E-6; dx=2.569963328966308E-7 evalInputDelta=-5.364749611500258E-7
New Minimum: 1.6343035499024183E-6 > 1.727983503705612E-7
WOLF (strong): th(14.50496510503994)=1.727983503705612E-7; dx=1.849280426616725E-8 evalInputDelta=1.461505199531857E-6
END: th(4.834988368346647)=7.627478878124629E-7; dx=-1.4050954815415185E-7 evalInputDelta=8.715556620899555E-7
Fitness changed from 1.6343035499024183E-6 to 1.727983503705612E-7
Iteration 8 complete. Error: 1.727983503705612E-7 Total: 0.0483; Orientation: 0.0004; Line Search: 0.0365
th(0)=1.727983503705612E-7;dx=-3.462165583287209E-8
New Minimum: 1.727983503705612E-7 > 1.1819235763140627E-8
WOLF (strong): th(10.416666666666671)=1.1819235763140627E-8; dx=3.7136658282454845E-9 evalInputDelta=1.6097911460742058E-7
END: th(5.208333333333336)=4.2393009653939645E-8; dx=-1.545399500231319E-8 evalInputDelta=1.3040534071662156E-7
Fitness changed from 1.727983503705612E-7 to 1.1819235763140627E-8
Iteration 9 complete. Error: 1.1819235763140627E-8 Total: 0.0386; Orientation: 0.0004; Line Search: 0.0295
th(0)=1.1819235763140627E-8;dx=-1.8837793672967996E-9
New Minimum: 1.1819235763140627E-8 > 1.3170165473531465E-9
WOLF (strong): th(11.221014010582733)=1.3170165473531465E-9; dx=1.1895203195039442E-11 evalInputDelta=1.050221921578748E-8
END: th(5.610507005291367)=3.909202290873565E-9; dx=-9.359420820511578E-10 evalInputDelta=7.910033472267061E-9
Fitness changed from 1.1819235763140627E-8 to 1.3170165473531465E-9
Iteration 10 complete. Error: 1.3170165473531465E-9 Total: 0.0347; Orientation: 0.0004; Line Search: 0.0265
th(0)=1.3170165473531465E-9;dx=-1.925905749172947E-10
New Minimum: 1.3170165473531465E-9 > 1.4805254107201482E-10
END: th(12.087470920866618)=1.4805254107201482E-10; dx=-8.268924642039139E-13 evalInputDelta=1.1689640062811318E-9
Fitness changed from 1.3170165473531465E-9 to 1.4805254107201482E-10
Iteration 11 complete. Error: 1.4805254107201482E-10 Total: 0.0277; Orientation: 0.0004; Line Search: 0.0173
Low gradient: 4.803595705819495E-6
th(0)=1.4805254107201482E-10;dx=-2.307453170496749E-11
Armijo: th(26.041666666666682)=2.344448147161075E-10; dx=2.9709458320815077E-11 evalInputDelta=-8.639227364409267E-11
New Minimum: 1.4805254107201482E-10 > 1.942579369548472E-11
WOLF (strong): th(13.020833333333341)=1.942579369548472E-11; dx=3.3174633078799E-12 evalInputDelta=1.286267473765301E-10
END: th(4.34027777777778)=6.69940954582332E-11; dx=-1.4277200033997461E-11 evalInputDelta=8.105844561378161E-11
Fitness changed from 1.4805254107201482E-10 to 1.942579369548472E-11
Iteration 12 complete. Error: 1.942579369548472E-11 Total: 0.0504; Orientation: 0.0007; Line Search: 0.0411
Low gradient: 1.9234596321685206E-6
th(0)=1.942579369548472E-11;dx=-3.69969695658186E-12
New Minimum: 1.942579369548472E-11 > 1.4801161698736703E-12
END: th(9.350845008818945)=1.4801161698736703E-12; dx=-1.3860375500345803E-13 evalInputDelta=1.7945677525611052E-11
Fitness changed from 1.942579369548472E-11 to 1.4801161698736703E-12
Iteration 13 complete. Error: 1.4801161698736703E-12 Total: 0.0408; Orientation: 0.0005; Line Search: 0.0230
Low gradient: 4.1804472550711295E-7
th(0)=1.4801161698736703E-12;dx=-1.747613925243174E-13
New Minimum: 1.4801161698736703E-12 > 1.3176369630762847E-13
WOLF (strong): th(20.145784868111033)=1.3176369630762847E-13; dx=4.0901879740646986E-14 evalInputDelta=1.3483524735660418E-12
END: th(10.072892434055516)=2.628516972110599E-13; dx=-6.692975639197846E-14 evalInputDelta=1.2172644726626103E-12
Fitness changed from 1.4801161698736703E-12 to 1.3176369630762847E-13
Iteration 14 complete. Error: 1.3176369630762847E-13 Total: 0.0426; Orientation: 0.0006; Line Search: 0.0334
Low gradient: 1.6381830998811918E-7
th(0)=1.3176369630762847E-13;dx=-2.6836438687363502E-14
Armijo: th(21.701388888888903)=2.277442049442656E-13; dx=3.568200236353111E-14 evalInputDelta=-9.598050863663715E-14
New Minimum: 1.3176369630762847E-13 > 1.0161825382774173E-14
WOLF (strong): th(10.850694444444452)=1.0161825382774173E-14; dx=4.422781838776449E-15 evalInputDelta=1.216018709248543E-13
END: th(3.6168981481481506)=5.3542600401778544E-14; dx=-1.6416698513895128E-14 evalInputDelta=7.822109590584992E-14
Fitness changed from 1.3176369630762847E-13 to 1.0161825382774173E-14
Iteration 15 complete. Error: 1.0161825382774173E-14 Total: 0.0517; Orientation: 0.0004; Line Search: 0.0410
Low gradient: 4.285315367598562E-8
th(0)=1.0161825382774173E-14;dx=-1.83639277997764E-15
New Minimum: 1.0161825382774173E-14 > 1.3867651273975932E-15
END: th(7.792370840682455
...skipping 5899 bytes...
2.649959361250642E-26 Total: 0.0456; Orientation: 0.0006; Line Search: 0.0353
Zero gradient: 7.441793702804576E-14
th(0)=2.649959361250642E-26;dx=-5.5380293515101834E-27
New Minimum: 2.649959361250642E-26 > 1.3910529267590447E-26
WOLF (strong): th(15.698342656893029)=1.3910529267590447E-26; dx=3.93421719551457E-27 evalInputDelta=1.258906434491597E-26
New Minimum: 1.3910529267590447E-26 > 1.619516432214587E-27
END: th(7.8491713284465146)=1.619516432214587E-27; dx=-8.020346917774494E-28 evalInputDelta=2.488007718029183E-26
Fitness changed from 2.649959361250642E-26 to 1.619516432214587E-27
Iteration 27 complete. Error: 1.619516432214587E-27 Total: 0.0326; Orientation: 0.0005; Line Search: 0.0234
Zero gradient: 1.4510936646649417E-14
th(0)=1.619516432214587E-27;dx=-2.10567282363073E-28
New Minimum: 1.619516432214587E-27 > 2.060092310737357E-28
WOLF (strong): th(16.910526998008816)=2.060092310737357E-28; dx=4.3304586945255803E-29 evalInputDelta=1.4135072011408513E-27
END: th(8.455263499004408)=3.7624986652848483E-28; dx=-8.377951902286869E-29 evalInputDelta=1.2432665656861021E-27
Fitness changed from 1.619516432214587E-27 to 2.060092310737357E-28
Iteration 28 complete. Error: 2.060092310737357E-28 Total: 0.0394; Orientation: 0.0003; Line Search: 0.0317
Zero gradient: 6.840575842150996E-15
th(0)=2.060092310737357E-28;dx=-4.6793477852219805E-29
Armijo: th(18.216312995615464)=2.37352954497244E-28; dx=5.024184982397422E-29 evalInputDelta=-3.134372342350832E-29
New Minimum: 2.060092310737357E-28 > 9.997662835476173E-31
WOLF (strong): th(9.108156497807732)=9.997662835476173E-31; dx=1.9433405829664614E-30 evalInputDelta=2.050094647901881E-28
END: th(3.0360521659359105)=8.79942626127621E-29; dx=-3.0524405507262056E-29 evalInputDelta=1.180149684609736E-28
Fitness changed from 2.060092310737357E-28 to 9.997662835476173E-31
Iteration 29 complete. Error: 9.997662835476173E-31 Total: 0.0387; Orientation: 0.0004; Line Search: 0.0305
Zero gradient: 4.0039029059607314E-16
th(0)=9.997662835476173E-31;dx=-1.603123848036079E-31
New Minimum: 9.997662835476173E-31 > 3.0031667599478296E-31
END: th(6.540976107038763)=3.0031667599478296E-31; dx=-6.655734704997547E-32 evalInputDelta=6.994496075528344E-31
Fitness changed from 9.997662835476173E-31 to 3.0031667599478296E-31
Iteration 30 complete. Error: 3.0031667599478296E-31 Total: 0.0207; Orientation: 0.0003; Line Search: 0.0138
Zero gradient: 1.9292946729905054E-16
th(0)=3.0031667599478296E-31;dx=-3.7221779352295404E-32
New Minimum: 3.0031667599478296E-31 > 3.140870750971517E-32
END: th(14.092105831674017)=3.140870750971517E-32; dx=-5.98652982425416E-34 evalInputDelta=2.689079684850678E-31
Fitness changed from 3.0031667599478296E-31 to 3.140870750971517E-32
Iteration 31 complete. Error: 3.140870750971517E-32 Total: 0.0225; Orientation: 0.0004; Line Search: 0.0154
Zero gradient: 7.540979452059138E-17
th(0)=3.140870750971517E-32;dx=-5.6866371096378144E-33
Armijo: th(30.36052165935911)=1.3508793618256136E-31; dx=1.216810627769622E-32 evalInputDelta=-1.0367922867284618E-31
New Minimum: 3.140870750971517E-32 > 1.2711137632955757E-32
WOLF (strong): th(15.180260829679556)=1.2711137632955757E-32; dx=3.183238641841444E-33 evalInputDelta=1.869756987675941E-32
New Minimum: 1.2711137632955757E-32 > 1.1940765655200862E-32
END: th(5.060086943226518)=1.1940765655200862E-32; dx=-2.457257038188742E-33 evalInputDelta=1.9467941854514305E-32
Fitness changed from 3.140870750971517E-32 to 1.1940765655200862E-32
Iteration 32 complete. Error: 1.1940765655200862E-32 Total: 0.0422; Orientation: 0.0004; Line Search: 0.0349
Zero gradient: 4.141987528359429E-17
th(0)=1.1940765655200862E-32;dx=-1.715606068508505E-33
New Minimum: 1.1940765655200862E-32 > 5.7777898331617076E-34
END: th(10.901626845064607)=5.7777898331617076E-34; dx=-8.706435943794712E-35 evalInputDelta=1.1362986671884692E-32
Fitness changed from 1.1940765655200862E-32 to 5.7777898331617076E-34
Iteration 33 complete. Error: 5.7777898331617076E-34 Total: 0.0225; Orientation: 0.0004; Line Search: 0.0154
Zero gradient: 8.317230435335941E-18
th(0)=5.7777898331617076E-34;dx=-6.917632211447847E-35
New Minimum: 5.7777898331617076E-34 > 8.024708101613482E-35
WOLF (strong): th(23.48684305279003)=8.024708101613482E-35; dx=1.8248571409057935E-35 evalInputDelta=4.9753190230003596E-34
END: th(11.743421526395014)=8.024708101613482E-35; dx=-1.8248571409057935E-35 evalInputDelta=4.9753190230003596E-34
Fitness changed from 5.7777898331617076E-34 to 8.024708101613482E-35
Iteration 34 complete. Error: 8.024708101613482E-35 Total: 0.0329; Orientation: 0.0004; Line Search: 0.0259
Zero gradient: 3.703575713270338E-18
th(0)=8.024708101613482E-35;dx=-1.3716473063925892E-35
Armijo: th(25.300434716132596)=1.2839532962581572E-34; dx=1.645976767671107E-35 evalInputDelta=-4.8148248609680896E-35
New Minimum: 8.024708101613482E-35 > 1.6049416203226965E-35
END: th(12.650217358066298)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=6.419766481290786E-35
Fitness changed from 8.024708101613482E-35 to 1.6049416203226965E-35
Iteration 35 complete. Error: 1.6049416203226965E-35 Total: 0.0294; Orientation: 0.0003; Line Search: 0.0218
Zero gradient: 1.656289410937949E-18
th(0)=1.6049416203226965E-35;dx=-2.7432946127851784E-36
Armijo: th(27.25406711266152)=6.419766481290786E-35; dx=5.486589225570357E-36 evalInputDelta=-4.8148248609680896E-35
Armijo: th(13.62703355633076)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(4.54234451877692)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(1.13558612969423)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(0.227117225938846)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(0.03785287098980767)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(0.005407552998543953)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(6.759441248179941E-4)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(7.51049027575549E-5)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(7.51049027575549E-6)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(6.827718432504991E-7)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(5.689765360420826E-8)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(4.3767425849390965E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
WOLFE (weak): th(3.126244703527926E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(2.3446835276459445E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(1.3286539989993685E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(8.206392346760806E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(5.666318525144366E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
WOLFE (weak): th(4.396281614336146E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(5.031300069740256E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
WOLFE (weak): th(4.713790842038201E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Armijo: th(4.872545455889228E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
WOLFE (weak): th(4.793168148963715E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
WOLFE (weak): th(4.832856802426471E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
mu ~= nu (4.832856802426471E-10): th(0.0)=1.6049416203226965E-35
Fitness changed from 1.6049416203226965E-35 to 1.6049416203226965E-35
Static Iteration Total: 0.2307; Orientation: 0.0003; Line Search: 0.2238
Iteration 36 failed. Error: 1.6049416203226965E-35
Previous Error: 0.0 -> 1.6049416203226965E-35
Optimization terminated 36
Final threshold in iteration 36: 1.6049416203226965E-35 (> 0.0) after 1.776s (< 30.000s)
Returns
1.6049416203226965E-35
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 2.81 seconds (0.175 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11409492533965
Reset training subject: 11409499724806
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=0.5923240879573332}, derivative=-0.11242199336048941}
New Minimum: 0.5923240879573332 > 0.592324087946091
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.592324087946091}, derivative=-0.11242199335933484}, evalInputDelta = -1.1242118347354335E-11
New Minimum: 0.592324087946091 > 0.5923240878786378
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.5923240878786378}, derivative=-0.11242199335240732}, evalInputDelta = -7.869538354299266E-11
New Minimum: 0.5923240878786378 > 0.5923240874064655
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.5923240874064655}, derivative=-0.11242199330391463}, evalInputDelta = -5.508676848009486E-10
New Minimum: 0.5923240874064655 > 0.5923240841012589
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.5923240841012589}, derivative=-0.1124219929644658}, evalInputDelta = -3.85607423769585E-9
New Minimum: 0.5923240841012589 > 0.5923240609648128
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.5923240609648128}, derivative=-0.11242199058832414}, evalInputDelta = -2.6992520330004766E-8
New Minimum: 0.5923240609648128 > 0.5923238990097053
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.5923238990097053}, derivative=-0.11242197395533243}, evalInputDelta = -1.8894762787713404E-7
New Minimum: 0.5923238990097053 > 0.5923227653246226
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.5923227653246226}, derivative=-0.11242185752439057}, evalInputDelta = -1.3226327105764213E-6
New Minimum: 0.5923227653246226 > 0.5923148295619188
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.5923148295619188}, derivative=-0.1124210425077975}, evalInputDelta = -9.258395414324383E-6
New Minimum: 0.5923148295619188 > 0.5922592808338755
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.5922592808338755}, derivative=-0.1124153373916459}, evalInputDelta = -6.480712345768858E-5
New Minimum: 0.5922592808338755 > 0.5918705186708335
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.5918705186708335}, derivative=-0.11237540157858478}, evalInputDelta = -4.5356928649964434E-4
New Minimum: 0.5918705186708335 > 0.5891530512593933
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.5891530512593933}, derivative=-0.11209585088715694}, evalInputDelta = -0.003171036697939833
New Minimum: 0.5891530512593933 > 0.5703202981421187
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.5703202981421187}, derivative=-0.11013899604716208}, evalInputDelta = -0.022003789815214447
New Minimum: 0.5703202981421187 > 0.4477774456986945
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.4477774456986945}, derivative=-0.09644101216719805}, evalInputDelta = -0.14454664225863867
New Minimum: 0.4477774456986945 > 0.045012028092125754
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.045012028092125754}, derivative=-5.551250074499476E-4}, evalInputDelta = -0.5473120598652074
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=19.522347030218782}, derivative=0.6706460851107869}, evalInputDelta = 18.93002294226145
F(5.217100560376924) = LineSearchPoint{point=PointSample{avg=0.16293589382093218}, derivative=-0.052185987324237404}, evalInputDelta = -0.42938819413640095
F(36.51970392263847) = LineSearchPoint{point=PointSample{avg=4.18601004678109}, derivative=0.3092300488932748}, evalInputDelta = 3.5936859588237566
F(2.809207994049113) = LineSearchPoint{point=PointSample{avg=0.3220653365715873}, derivative=-0.07998722087943064}, evalInputDelta = -0.27025875138574584
F(19.66445595834379) = LineSearchPoint{point=PointSample{avg=0.613949293175157}, derivative=0.1146214140069221}, evalInputDelta = 0.02162520521782385
F(1.5126504583341378) = LineSearchPoint{point=PointSample{avg=0.4354780356435999}, derivative=-0.09495711587068853}, evalInputDelta = -0.15684605231373328
F(10.588553208338965) = LineSearchPoint{point=PointSample{avg=0.0491850754948346}, derivative=0.00983214906811677}, evalInputDelta = -0.5431390124624985
0.0491850754948346 <= 0.5923240879573332
New Minimum: 0.045012028092125754 > 0.04499868290622055
F(9.736980971260193) = LineSearchPoint{point=PointSample{avg=0.04499868290622055}, derivative=-2.054563116860031E-17}, evalInputDelta = -0.5473254050511126
Left bracket at 9.736980971260193
Converged to left
Fitness changed from 0.5923240879573332 to 0.04499868290622055
Iteration 1 complete. Error: 0.04499868290622055 Total: 0.2574; Orientation: 0.0003; Line Search: 0.2370
F(0.0) = LineSearchPoint{point=PointSample{avg=0.04499868290622055}, derivative=-0.005673757518481185}
New Minimum: 0.04499868290622055 > 0.008103756586050434
F(9.736980971260193) = LineSearchPoint{point=PointSample{avg=0.008103756586050434}, derivative=-0.0019045516985275956}, evalInputDelta = -0.036894926320170114
F(68.15886679882135) = LineSearchPoint{point=PointSample{avg=0.5574485909237065}, derivative=0.02071068322119394}, evalInputDelta = 0.5124499080174859
F(5.242989753755489) = LineSearchPoint{point=PointSample{avg=0.020571743927556892}, derivative=-0.0036441851538907893}, evalInputDelta = -0.024426938978663656
F(36.70092827628842) = LineSearchPoint{point=PointSample{avg=0.09747167940868244}, derivative=0.008533249033651574}, evalInputDelta = 0.05247299650246189
F(2.8231483289452632) = LineSearchPoint{point=PointSample{avg=0.030523457957040066}, derivative=-0.0045809108606248185}, evalInputDelta = -0.014475224949180483
F(19.762038302616844) = LineSearchPoint{point=PointSample{avg=0.008462740796290173}, derivative=0.0019761690865133783}, evalInputDelta = -0.03653594210993037
0.008462740796290173 <= 0.04499868290622055
New Minimum: 0.008103756586050434 > 0.0034185364135325166
F(14.657005117771995) = LineSearchPoint{point=PointSample{avg=0.0034185364135325166}, derivative=-4.87890977618477E-19}, evalInputDelta = -0.041580146492688035
Left bracket at 14.657005117771995
Converged to left
Fitness changed from 0.04499868290622055 to 0.0034185364135325166
Iteration 2 complete. Error: 0.0034185364135325166 Total: 0.1029; Orientation: 0.0005; Line Search: 0.0929
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0034185364135325166}, derivative=-6.488317557877665E-4}
New Minimum: 0.0034185364135325166 > 0.0010662198119841457
F(14.657005117771995) = LineSearchPoint{point=PointSample{avg=0.0010662198119841457}, derivative=3.278498658795581E-4}, evalInputDelta = -0.002352316601548371
0.0010662198119841457 <= 0.0034185364135325166
New Minimum: 0.0010662198119841457 > 2.597051837051037E-4
F(9.736980971260188) = LineSearchPoint{point=PointSample{avg=2.597051837051037E-4}, derivative=-2.371692252312041E-19}, evalInputDelta = -0.003158831229827413
Left bracket at 9.736980971260188
Converged to left
Fitness changed from 0.0034185364135325166 to 2.597051837051037E-4
Iteration 3 complete. Error: 2.597051837051037E-4 Total: 0.0510; Orientation: 0.0005; Line Search: 0.0418
F(0.0) = LineSearchPoint{point=PointSample{avg=2.597051837051037E-4}, derivative=-3.2745497056129924E-5}
New Minimum: 2.597051837051037E-4 > 4.676998207409209E-5
F(9.736980971260188) = LineSearchPoint{point=PointSample{avg=4.676998207409209E-5}, derivative=-1.099192058071557E-5}, evalInputDelta = -2.1293520163101162E-4
F(68.15886679882132) = LineSearchPoint{point=PointSample{avg=0.0032172561364452487}, derivative=1.1952953827177085E-4}, evalInputDelta = 0.0029575509527401452
F(5.242989753755486) = LineSearchPoint{point=PointSample{avg=1.1872766469575891E-4}, derivative=-2.1032032800137563E-5}, evalInputDelta = -1.4097751900934478E-4
F(36.7009282762884) = LineSearchPoint{point=PointSample{avg=5.625475852178189E-4}, derivative=4.924875273581661E-5}, evalInputDelta = 3.028424015127152E-4
F(2.8231483289452615) = LineSearchPoint{point=PointSample{avg=1.761629395368877E-4}, derivative=-2.6438247072134013E-5}, evalInputDelta = -8.354224416821601E-5
F(19.76203830261683) = LineSearchPoint{point=PointSample{avg=4.884182183130873E-5}, derivative=1.1405252831841275E-5}, evalInputDelta = -2.10863361
...skipping 28054 bytes...
4907E-30 Total: 0.0797; Orientation: 0.0004; Line Search: 0.0724
Zero gradient: 9.446677794110383E-16
F(0.0) = LineSearchPoint{point=PointSample{avg=4.697535727354907E-30}, derivative=-8.92397213457382E-31}
New Minimum: 4.697535727354907E-30 > 1.5461526087702761E-30
F(14.626390146564637) = LineSearchPoint{point=PointSample{avg=1.5461526087702761E-30}, derivative=4.685598274299209E-31}, evalInputDelta = -3.151383118584631E-30
1.5461526087702761E-30 <= 4.697535727354907E-30
New Minimum: 1.5461526087702761E-30 > 3.6997114231678805E-31
F(9.59071404724496) = LineSearchPoint{point=PointSample{avg=3.6997114231678805E-31}, derivative=-3.8261910944756658E-34}, evalInputDelta = -4.327564585038119E-30
Left bracket at 9.59071404724496
Converged to left
Fitness changed from 4.697535727354907E-30 to 3.6997114231678805E-31
Iteration 27 complete. Error: 3.6997114231678805E-31 Total: 0.0391; Orientation: 0.0004; Line Search: 0.0312
Zero gradient: 2.161405140879455E-16
F(0.0) = LineSearchPoint{point=PointSample{avg=3.6997114231678805E-31}, derivative=-4.6716721830201366E-32}
New Minimum: 3.6997114231678805E-31 > 6.710260914569194E-32
F(9.59071404724496) = LineSearchPoint{point=PointSample{avg=6.710260914569194E-32}, derivative=-1.374664237131873E-32}, evalInputDelta = -3.028685331710961E-31
F(67.13499833071472) = LineSearchPoint{point=PointSample{avg=4.453408057487621E-30}, derivative=1.6815912028696718E-31}, evalInputDelta = 4.083436915170833E-30
F(5.16423064082421) = LineSearchPoint{point=PointSample{avg=1.8323618479224224E-31}, derivative=-2.8480135868373337E-32}, evalInputDelta = -1.8673495752454582E-31
F(36.14961448576947) = LineSearchPoint{point=PointSample{avg=8.293054340531438E-31}, derivative=7.108944539510752E-32}, evalInputDelta = 4.5933429173635575E-31
F(2.7807395758284206) = LineSearchPoint{point=PointSample{avg=2.490387912254728E-31}, derivative=-3.7429442363362975E-32}, evalInputDelta = -1.2093235109131524E-31
F(19.465177030798944) = LineSearchPoint{point=PointSample{avg=7.280015189783752E-32}, derivative=1.6387309569971356E-32}, evalInputDelta = -2.9717099041895054E-31
7.280015189783752E-32 <= 3.6997114231678805E-31
New Minimum: 6.710260914569194E-32 > 3.158525108795067E-32
F(14.410319603146151) = LineSearchPoint{point=PointSample{avg=3.158525108795067E-32}, derivative=-2.7502464495124373E-33}, evalInputDelta = -3.383858912288374E-31
Left bracket at 14.410319603146151
F(15.136750066839637) = LineSearchPoint{point=PointSample{avg=4.911121358187451E-32}, derivative=4.111724845798664E-33}, evalInputDelta = -3.2085992873491354E-31
Right bracket at 15.136750066839637
New Minimum: 3.158525108795067E-32 > 2.824697251767946E-32
F(14.701469582883878) = LineSearchPoint{point=PointSample{avg=2.824697251767946E-32}, derivative=-1.7686692903362623E-33}, evalInputDelta = -3.4172416979910858E-31
Left bracket at 14.701469582883878
Converged to left
Fitness changed from 3.6997114231678805E-31 to 2.824697251767946E-32
Iteration 28 complete. Error: 2.824697251767946E-32 Total: 0.1394; Orientation: 0.0005; Line Search: 0.1314
Zero gradient: 7.116780725909866E-17
F(0.0) = LineSearchPoint{point=PointSample{avg=2.824697251767946E-32}, derivative=-5.0648567900682174E-33}
New Minimum: 2.824697251767946E-32 > 1.502225356622044E-32
F(14.701469582883878) = LineSearchPoint{point=PointSample{avg=1.502225356622044E-32}, derivative=3.204704286629606E-33}, evalInputDelta = -1.3224718951459018E-32
1.502225356622044E-32 <= 2.824697251767946E-32
New Minimum: 1.502225356622044E-32 > 3.723464559148656E-33
F(9.004206795287889) = LineSearchPoint{point=PointSample{avg=3.723464559148656E-33}, derivative=9.974776367970375E-35}, evalInputDelta = -2.4523507958530802E-32
Right bracket at 9.004206795287889
F(8.830302001185546) = LineSearchPoint{point=PointSample{avg=3.723464559148656E-33}, derivative=9.974776367970375E-35}, evalInputDelta = -2.4523507958530802E-32
Right bracket at 8.830302001185546
New Minimum: 3.723464559148656E-33 > 2.6963019221421302E-33
F(8.659755956843103) = LineSearchPoint{point=PointSample{avg=2.6963019221421302E-33}, derivative=-3.124813631048512E-34}, evalInputDelta = -2.5550670595537327E-32
Left bracket at 8.659755956843103
Converged to right
Fitness changed from 2.824697251767946E-32 to 2.6963019221421302E-33
Iteration 29 complete. Error: 2.6963019221421302E-33 Total: 0.0951; Orientation: 0.0004; Line Search: 0.0857
Zero gradient: 2.0767551877920566E-17
F(0.0) = LineSearchPoint{point=PointSample{avg=2.6963019221421302E-33}, derivative=-4.31291211002122E-34}
New Minimum: 2.6963019221421302E-33 > 3.8518598887744713E-34
F(8.830302001185546) = LineSearchPoint{point=PointSample{avg=3.8518598887744713E-34}, derivative=-5.462553619864404E-35}, evalInputDelta = -2.311115933264683E-33
F(61.81211400829882) = LineSearchPoint{point=PointSample{avg=5.071615520219721E-32}, derivative=1.951571005294829E-33}, evalInputDelta = 4.8019853280055077E-32
F(4.754778000638371) = LineSearchPoint{point=PointSample{avg=6.419766481290786E-34}, derivative=-8.014950056561918E-35}, evalInputDelta = -2.0543252740130518E-33
F(33.2834460044686) = LineSearchPoint{point=PointSample{avg=8.490141171507064E-33}, derivative=7.460559566490387E-34}, evalInputDelta = 5.7938392493649336E-33
F(2.5602650772668154) = LineSearchPoint{point=PointSample{avg=2.6963019221421302E-33}, derivative=-4.31291211002122E-34}, evalInputDelta = 0.0
F(17.921855540867707) = LineSearchPoint{point=PointSample{avg=1.300002712461384E-33}, derivative=1.5374467634449975E-34}, evalInputDelta = -1.3962992096807462E-33
1.300002712461384E-33 <= 2.6963019221421302E-33
F(13.212076296178273) = LineSearchPoint{point=PointSample{avg=1.348150961071065E-33}, derivative=1.3191849747074809E-34}, evalInputDelta = -1.3481509610710653E-33
1.348150961071065E-33 > 1.300002712461384E-33
Fitness changed from 2.6963019221421302E-33 to 3.8518598887744713E-34
Iteration 30 complete. Error: 3.8518598887744713E-34 Total: 0.0907; Orientation: 0.0005; Line Search: 0.0809
Zero gradient: 7.14478332309317E-18
F(0.0) = LineSearchPoint{point=PointSample{avg=3.8518598887744713E-34}, derivative=-5.10479287339503E-35}
New Minimum: 3.8518598887744713E-34 > 6.419766481290786E-35
F(17.921855540867707) = LineSearchPoint{point=PointSample{avg=6.419766481290786E-35}, derivative=1.0973178451140713E-35}, evalInputDelta = -3.209883240645393E-34
6.419766481290786E-35 <= 3.8518598887744713E-34
F(14.7510040686647) = LineSearchPoint{point=PointSample{avg=6.419766481290786E-35}, derivative=1.0973178451140713E-35}, evalInputDelta = -3.209883240645393E-34
Right bracket at 14.7510040686647
F(12.14116030215628) = LineSearchPoint{point=PointSample{avg=1.4444474582904269E-34}, derivative=-5.366411197040591E-36}, evalInputDelta = -2.4074124304840444E-34
Left bracket at 12.14116030215628
F(12.9983112534763) = LineSearchPoint{point=PointSample{avg=2.0864241064195054E-34}, derivative=5.606767254100122E-36}, evalInputDelta = -1.765435782354966E-34
2.0864241064195054E-34 > 6.419766481290786E-35
Fitness changed from 3.8518598887744713E-34 to 6.419766481290786E-35
Iteration 31 complete. Error: 6.419766481290786E-35 Total: 0.0733; Orientation: 0.0004; Line Search: 0.0653
Zero gradient: 3.312578821875898E-18
F(0.0) = LineSearchPoint{point=PointSample{avg=6.419766481290786E-35}, derivative=-1.0973178451140713E-35}
F(14.7510040686647) = LineSearchPoint{point=PointSample{avg=6.419766481290786E-35}, derivative=1.0973178451140713E-35}, evalInputDelta = 0.0
6.419766481290786E-35 <= 6.419766481290786E-35
New Minimum: 6.419766481290786E-35 > 0.0
F(7.37550203433235) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -6.419766481290786E-35
Right bracket at 7.37550203433235
Converged to right
Fitness changed from 6.419766481290786E-35 to 0.0
Iteration 32 complete. Error: 0.0 Total: 0.0501; Orientation: 0.0003; Line Search: 0.0429
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0154; Orientation: 0.0003; Line Search: 0.0075
Iteration 33 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 33
Final threshold in iteration 33: 0.0 (> 0.0) after 2.808s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 2.68 seconds (0.181 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11412308767179
Reset training subject: 11412317905091
Adding measurement 417027f2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 0.5923240879573332 < 0.5923240879573332. Total: 1
th(0)=0.5923240879573332;dx=-0.11242199336048941
Adding measurement 68f6ba80 to history. Total: 1
New Minimum: 0.5923240879573332 > 0.37691385449489573
END: th(2.154434690031884)=0.37691385449489573; dx=-0.08754715349654488 evalInputDelta=0.21541023346243743
Fitness changed from 0.5923240879573332 to 0.37691385449489573
Iteration 1 complete. Error: 0.37691385449489573 Total: 0.0522; Orientation: 0.0013; Line Search: 0.0237
Non-optimal measurement 0.37691385449489573 < 0.37691385449489573. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.37691385449489573 < 0.37691385449489573. Total: 2
th(0)=0.37691385449489573;dx=-0.06845397026813493
Adding measurement 4ea3ec34 to history. Total: 2
New Minimum: 0.37691385449489573 > 0.13264409262698712
END: th(4.641588833612779)=0.13264409262698712; dx=-0.03679867947067503 evalInputDelta=0.2442697618679086
Fitness changed from 0.37691385449489573 to 0.13264409262698712
Iteration 2 complete. Error: 0.13264409262698712 Total: 0.0324; Orientation: 0.0011; Line Search: 0.0214
Non-optimal measurement 0.13264409262698712 < 0.13264409262698712. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 0.13264409262698712 < 0.13264409262698712. Total: 3
th(0)=0.13264409262698712;dx=-0.02070918776217733
Adding measurement 52873c96 to history. Total: 3
New Minimum: 0.13264409262698712 > 0.016658739614842474
END: th(10.000000000000002)=0.016658739614842474; dx=-0.0024878828402515934 evalInputDelta=0.11598535301214465
Fitness changed from 0.13264409262698712 to 0.016658739614842474
Iteration 3 complete. Error: 0.016658739614842474 Total: 0.0322; Orientation: 0.0011; Line Search: 0.0214
Non-optimal measurement 0.016658739614842474 < 0.016658739614842474. Total: 4
Rejected: LBFGS Orientation magnitude: 5.283e-01, gradient 4.476e-02, dot -0.981; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.016658739614842474, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
LBFGS Accumulation History: 3 points
Removed measurement 52873c96 to history. Total: 3
Adding measurement 6dafd1e1 to history. Total: 3
th(0)=0.016658739614842474;dx=-0.0020034047732510963
Adding measurement 45d6af17 to history. Total: 4
New Minimum: 0.016658739614842474 > 0.0029055153861502984
WOLF (strong): th(21.544346900318843)=0.0029055153861502984; dx=7.266685331296899E-4 evalInputDelta=0.013753224228692176
Adding measurement 6b13d587 to history. Total: 5
New Minimum: 0.0029055153861502984 > 0.002429921703500375
END: th(10.772173450159421)=0.002429921703500375; dx=-6.383681200607035E-4 evalInputDelta=0.0142288179113421
Fitness changed from 0.016658739614842474 to 0.002429921703500375
Iteration 4 complete. Error: 0.002429921703500375 Total: 0.0630; Orientation: 0.0223; Line Search: 0.0307
Non-optimal measurement 0.002429921703500375 < 0.002429921703500375. Total: 6
Rejected: LBFGS Orientation magnitude: 2.911e-01, gradient 1.678e-02, dot -0.995; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.002429921703500375, 0.0029055153861502984, 0.016658739614842474, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
Rejected: LBFGS Orientation magnitude: 2.912e-01, gradient 1.678e-02, dot -0.995; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.002429921703500375, 0.0029055153861502984, 0.016658739614842474, 0.13264409262698712, 0.37691385449489573
Rejected: LBFGS Orientation magnitude: 2.913e-01, gradient 1.678e-02, dot -0.995; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.002429921703500375, 0.0029055153861502984, 0.016658739614842474, 0.13264409262698712
LBFGS Accumulation History: 3 points
Removed measurement 6b13d587 to history. Total: 5
Removed measurement 45d6af17 to history. Total: 4
Removed measurement 6dafd1e1 to history. Total: 3
Adding measurement 71e94a15 to history. Total: 3
th(0)=0.002429921703500375;dx=-2.8153468095580157E-4
Adding measurement b1c1035 to history. Total: 4
New Minimum: 0.002429921703500375 > 3.817075835563029E-4
WOLF (strong): th(23.2079441680639)=3.817075835563029E-4; dx=1.0502493885097621E-4 evalInputDelta=0.002048214119944072
Adding measurement 17ff4f29 to history. Total: 5
New Minimum: 3.817075835563029E-4 > 2.844078842653764E-4
END: th(11.60397208403195)=2.844078842653764E-4; dx=-8.825487105241288E-5 evalInputDelta=0.0021455138192349983
Fitness changed from 0.002429921703500375 to 2.844078842653764E-4
Iteration 5 complete. Error: 2.844078842653764E-4 Total: 0.0901; Orientation: 0.0510; Line Search: 0.0315
Non-optimal measurement 2.844078842653764E-4 < 2.844078842653764E-4. Total: 6
Rejected: LBFGS Orientation magnitude: 1.006e-01, gradient 5.735e-03, dot -0.995; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.844078842653764E-4, 3.817075835563029E-4, 0.002429921703500375, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
Rejected: LBFGS Orientation magnitude: 1.017e-01, gradient 5.735e-03, dot -0.995; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.844078842653764E-4, 3.817075835563029E-4, 0.002429921703500375, 0.13264409262698712, 0.37691385449489573
Rejected: LBFGS Orientation magnitude: 1.150e-01, gradient 5.735e-03, dot -1.000; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.844078842653764E-4, 3.817075835563029E-4, 0.002429921703500375, 0.13264409262698712
LBFGS Accumulation History: 3 points
Removed measurement 17ff4f29 to history. Total: 5
Removed measurement b1c1035 to history. Total: 4
Removed measurement 71e94a15 to history. Total: 3
Adding measurement 6b0f1b9e to history. Total: 3
th(0)=2.844078842653764E-4;dx=-3.2885330336707554E-5
Adding measurement 5f5021c to history. Total: 4
New Minimum: 2.844078842653764E-4 > 6.79426619810632E-5
WOLF (strong): th(25.000000000000007)=6.79426619810632E-5; dx=1.5568112553962553E-5 evalInputDelta=2.1646522228431318E-4
Adding measurement 22803c26 to history. Total: 5
New Minimum: 6.79426619810632E-5 > 2.4758264089874862E-5
END: th(12.500000000000004)=2.4758264089874862E-5; dx=-8.658608891372419E-6 evalInputDelta=2.5964962017550154E-4
Fitness changed from 2.844078842653764E-4 to 2.4758264089874862E-5
Iteration 6 complete. Error: 2.4758264089874862E-5 Total: 0.1152; Orientation: 0.0764; Line Search: 0.0307
Non-optimal measurement 2.4758264089874862E-5 < 2.4758264089874862E-5. Total: 6
Rejected: LBFGS Orientation magnitude: 2.935e-02, gradient 1.710e-03, dot -0.990; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.4758264089874862E-5, 6.79426619810632E-5, 2.844078842653764E-4, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
Rejected: LBFGS Orientation magnitude: 2.944e-02, gradient 1.710e-03, dot -0.989; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.4758264089874862E-5, 6.79426619810632E-5, 2.844078842653764E-4, 0.13264409262698712, 0.37691385449489573
Rejected: LBFGS Orientation magnitude: 3.017e-02, gradient 1.710e-03, dot -0.984; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 1.000/1.000e+00, ebe
...skipping 45558 bytes...
E-35
Adding measurement 42ba994c to history. Total: 4
New Minimum: 5.7777898331617076E-34 > 8.024708101613482E-35
WOLF (strong): th(23.48684305279003)=8.024708101613482E-35; dx=1.8248571409057935E-35 evalInputDelta=4.9753190230003596E-34
Non-optimal measurement 8.024708101613482E-35 < 8.024708101613482E-35. Total: 5
END: th(11.743421526395014)=8.024708101613482E-35; dx=-1.8248571409057935E-35 evalInputDelta=4.9753190230003596E-34
Fitness changed from 5.7777898331617076E-34 to 8.024708101613482E-35
Iteration 34 complete. Error: 8.024708101613482E-35 Total: 0.0557; Orientation: 0.0279; Line Search: 0.0215
Non-optimal measurement 8.024708101613482E-35 < 8.024708101613482E-35. Total: 5
Rejected: LBFGS Orientation magnitude: 4.090e-17, gradient 3.704e-18, dot -0.943; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 0.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.024708101613482E-35, 5.7777898331617076E-34, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
Rejected: LBFGS Orientation magnitude: 4.108e-17, gradient 3.704e-18, dot -0.946; [ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00, c3c31259-3d78-4d96-9eeb-386eec83cd7b = 0.000e+00]
Orientation rejected. Popping history element from 8.024708101613482E-35, 5.7777898331617076E-34, 0.13264409262698712, 0.37691385449489573
LBFGS Accumulation History: 3 points
Removed measurement 42ba994c to history. Total: 4
Removed measurement 3e80cf2d to history. Total: 3
Adding measurement 3a20db9d to history. Total: 3
th(0)=8.024708101613482E-35;dx=-1.3716473063925892E-35
Non-optimal measurement 1.2839532962581572E-34 < 8.024708101613482E-35. Total: 4
Armijo: th(25.300434716132596)=1.2839532962581572E-34; dx=1.645976767671107E-35 evalInputDelta=-4.8148248609680896E-35
Adding measurement 5a3a5aeb to history. Total: 4
New Minimum: 8.024708101613482E-35 > 1.6049416203226965E-35
END: th(12.650217358066298)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=6.419766481290786E-35
Fitness changed from 8.024708101613482E-35 to 1.6049416203226965E-35
Iteration 35 complete. Error: 1.6049416203226965E-35 Total: 0.0609; Orientation: 0.0299; Line Search: 0.0237
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 5
Rejected: LBFGS Orientation magnitude: 1.669e-17, gradient 1.656e-18, dot -0.981; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 0.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.6049416203226965E-35, 8.024708101613482E-35, 0.13264409262698712, 0.37691385449489573, 0.5923240879573332
Rejected: LBFGS Orientation magnitude: 1.698e-17, gradient 1.656e-18, dot -0.981; [c3c31259-3d78-4d96-9eeb-386eec83cd7b = 0.000e+00, ebeefa11-0e61-4e9e-b33f-c4aaa8ec83ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.6049416203226965E-35, 8.024708101613482E-35, 0.13264409262698712, 0.37691385449489573
LBFGS Accumulation History: 3 points
Removed measurement 5a3a5aeb to history. Total: 4
Removed measurement 3a20db9d to history. Total: 3
Adding measurement 55eda591 to history. Total: 3
th(0)=1.6049416203226965E-35;dx=-2.7432946127851784E-36
Non-optimal measurement 6.419766481290786E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(27.25406711266152)=6.419766481290786E-35; dx=5.486589225570357E-36 evalInputDelta=-4.8148248609680896E-35
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(13.62703355633076)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(4.54234451877692)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(1.13558612969423)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(0.227117225938846)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(0.03785287098980767)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(0.005407552998543953)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(6.759441248179941E-4)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(7.51049027575549E-5)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(7.51049027575549E-6)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(6.827718432504991E-7)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(5.689765360420826E-8)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(4.3767425849390965E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
WOLFE (weak): th(3.126244703527926E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(2.3446835276459445E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(1.3286539989993685E-9)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(8.206392346760806E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(5.666318525144366E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
WOLFE (weak): th(4.396281614336146E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(5.031300069740256E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
WOLFE (weak): th(4.713790842038201E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
Armijo: th(4.872545455889228E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
WOLFE (weak): th(4.793168148963715E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
WOLFE (weak): th(4.832856802426471E-10)=1.6049416203226965E-35; dx=-2.7432946127851784E-36 evalInputDelta=0.0
Non-optimal measurement 1.6049416203226965E-35 < 1.6049416203226965E-35. Total: 4
mu ~= nu (4.832856802426471E-10): th(0.0)=1.6049416203226965E-35
Fitness changed from 1.6049416203226965E-35 to 1.6049416203226965E-35
Static Iteration Total: 0.2032; Orientation: 0.0279; Line Search: 0.1695
Iteration 36 failed. Error: 1.6049416203226965E-35
Previous Error: 0.0 -> 1.6049416203226965E-35
Optimization terminated 36
Final threshold in iteration 36: 1.6049416203226965E-35 (> 0.0) after 2.676s (< 30.000s)
Returns
1.6049416203226965E-35
TrainingTester.java:432 executed in 0.13 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -34.79454076041352], [35.0, -0.42375789858091484]; valueStats=DoubleSummaryStatistics{count=101, sum=1.106613, min=0.000000, average=0.010957, max=0.376914}
Plotting 35 points for GD
Plotting 32 points for CjGD
Plotting 35 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -34.79454076041352], [2.535, -0.42375789858091484]; valueStats=DoubleSummaryStatistics{count=101, sum=1.106613, min=0.000000, average=0.010957, max=0.376914}
Plotting 35 points for GD
Plotting 32 points for CjGD
Plotting 35 points for LBFGS
Returns
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:279 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0037 = temp_18_0035.state();
assert temp_18_0037 != null;
String temp_18_0036 = temp_18_0037.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0037.freeRef();
return temp_18_0036;
Returns
We simultaneously regress this target input:
TrainingTester.java:287 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(x -> {
RefStream<Tensor> temp_18_0006 = RefArrays.stream(RefUtil.addRef(x));
if (null != x)
RefUtil.freeRef(x);
return temp_18_0006;
}).map(x -> {
String temp_18_0007 = x.prettyPrint();
x.freeRef();
return temp_18_0007;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.08, -0.128, 0.7 ] ]
]
[
[ [ 0.7, -0.128, 0.08 ] ]
]
Which produces the following output:
TrainingTester.java:308 executed in 0.00 seconds (0.000 gc):
return RefStream.of(RefUtil.addRef(output_target)).map(x -> {
String temp_18_0008 = x.prettyPrint();
x.freeRef();
return temp_18_0008;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.2795999999999999, 1.116864, -0.08921599999999999, 0.43111999999999995, -0.614448, 0.34649599999999997 ] ]
]
[
[ [ 0.9975999999999998, -0.12313599999999997, 0.825904, -0.37984, 0.04027200000000001, 1.1326559999999999 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.03 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11415272867890
Reset training subject: 11415283228878
Error java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:57)
at com.simiacryptus.mindseye.eval.BatchedTrainable.measure(BatchedTrainable.java:66)
at com.simiacryptus.mindseye.opt.IterativeTrainer.measure(IterativeTrainer.java:143)
at com.simiacryptus.mindseye.opt.IterativeTrainer.run(IterativeTrainer.java:202)
at com.simiacryptus.mindseye.test.unit.TrainingTester.lambda$trainGD$892d3ca8$1(TrainingTester.java:489)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainGD(TrainingTester.java:480)
at com.simiacryptus.mindseye.test.unit.TrainingTester.runOpt(TrainingTester.java:658)
at com.simiacryptus.mindseye.test.unit.TrainingTester.train(TrainingTester.java:606)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainAll(TrainingTester.java:414)
at com.simiacryptus.mindseye.test.unit.TrainingTester.testCompleteLearning(TrainingTester.java:318)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:247)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:58)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTe
...skipping 5922 bytes...
er.java:266)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.BinarySumLayer$Accumulator.lambda$accept$1(BinarySumLayer.java:324)
... 142 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 158 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy51.run(Unknown Source)
at com.simiacryptus.util.Util.lambda$runAllParallel$4(Util.java:399)
at com.simiacryptus.ref.wrappers.RefStream.lambda$forEach$25(RefStream.java:354)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at com.simiacryptus.ref.wrappers.StreamWrapper.forEach(StreamWrapper.java:129)
at com.simiacryptus.ref.wrappers.RefStream.forEach(RefStream.java:354)
at com.simiacryptus.util.Util.runAllParallel(Util.java:397)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:749)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:616)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.lambda$accept$2(ImgConcatLayer.java:381)
... 162 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 190 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.lang.ResourcePool.apply(ResourcePool.java:91)
at com.simiacryptus.mindseye.lang.cudnn.CudaSystem.run(CudaSystem.java:655)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$accept$3(SimpleConvolutionLayer.java:732)
... 194 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 198 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 6 more
Caused by: java.lang.IllegalStateException: Object com.simiacryptus.mindseye.lang.cudnn.CudaTensor (0 refs; 1 adds, 0 frees) apply current stack
at com.simiacryptus.ref.lang.ReferenceCountingBase.getStackTrace(ReferenceCountingBase.java:85)
at com.simiacryptus.ref.lang.ReferenceCountingBase.referenceReport(ReferenceCountingBase.java:325)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
... 10 more
Final threshold in iteration 1: 0.8831906026666665 (> 0.0) after 0.028s (< 30.000s)
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:57)
at com.simiacryptus.mindseye.eval.BatchedTrainable.measure(BatchedTrainable.java:66)
at com.simiacryptus.mindseye.opt.IterativeTrainer.measure(IterativeTrainer.java:143)
at com.simiacryptus.mindseye.opt.IterativeTrainer.run(IterativeTrainer.java:202)
at com.simiacryptus.mindseye.test.unit.TrainingTester.lambda$trainGD$892d3ca8$1(TrainingTester.java:489)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainGD(TrainingTester.java:480)
at com.simiacryptus.mindseye.test.unit.TrainingTester.runOpt(TrainingTester.java:658)
at com.simiacryptus.mindseye.test.unit.TrainingTester.train(TrainingTester.java:606)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainAll(TrainingTester.java:414)
at com.simiacryptus.mindseye.test.unit.TrainingTester.testCompleteLearning(TrainingTester.java:318)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:247)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:58)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambda$nu
...skipping 5754 bytes...
om.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.accept(ImgConcatLayer.java:266)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.BinarySumLayer$Accumulator.lambda$accept$1(BinarySumLayer.java:324)
... 142 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 158 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy51.run(Unknown Source)
at com.simiacryptus.util.Util.lambda$runAllParallel$4(Util.java:399)
at com.simiacryptus.ref.wrappers.RefStream.lambda$forEach$25(RefStream.java:354)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at com.simiacryptus.ref.wrappers.StreamWrapper.forEach(StreamWrapper.java:129)
at com.simiacryptus.ref.wrappers.RefStream.forEach(RefStream.java:354)
at com.simiacryptus.util.Util.runAllParallel(Util.java:397)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:749)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:616)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.lambda$accept$2(ImgConcatLayer.java:381)
... 162 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 190 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.lang.ResourcePool.apply(ResourcePool.java:91)
at com.simiacryptus.mindseye.lang.cudnn.CudaSystem.run(CudaSystem.java:655)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$accept$3(SimpleConvolutionLayer.java:732)
... 194 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 198 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 6 more
Caused by: java.lang.IllegalStateException: Object com.simiacryptus.mindseye.lang.cudnn.CudaTensor (0 refs; 1 adds, 0 frees) apply current stack
at com.simiacryptus.ref.lang.ReferenceCountingBase.getStackTrace(ReferenceCountingBase.java:85)
at com.simiacryptus.ref.lang.ReferenceCountingBase.referenceReport(ReferenceCountingBase.java:325)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
... 10 more
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[0.048, -0.852, 1.912, -0.384, 1.616, 1.524, -1.72, -0.128, 1.208]
[0.08, 0.7, 1.764, 1.108, -1.028, 0.496, -0.608, -0.804, 0.028]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.08, -0.128, 0.7 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 1.451296, 0.829232, 0.7243839999999999, 1.1516, 0.5674239999999999, 0.07387200000000001 ] ]
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.02 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11415314287044
Reset training subject: 11415327772334
Error java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:57)
at com.simiacryptus.mindseye.eval.BatchedTrainable.measure(BatchedTrainable.java:66)
at com.simiacryptus.mindseye.opt.IterativeTrainer.measure(IterativeTrainer.java:143)
at com.simiacryptus.mindseye.opt.IterativeTrainer.run(IterativeTrainer.java:193)
at com.simiacryptus.mindseye.test.unit.TrainingTester.lambda$trainCjGD$892d3ca8$1(TrainingTester.java:461)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainCjGD(TrainingTester.java:452)
at com.simiacryptus.mindseye.test.unit.TrainingTester.runOpt(TrainingTester.java:658)
at com.simiacryptus.mindseye.test.unit.TrainingTester.train(TrainingTester.java:606)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainAll(TrainingTester.java:416)
at com.simiacryptus.mindseye.test.unit.TrainingTester.testCompleteLearning(TrainingTester.java:318)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:247)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:58)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.r
...skipping 5926 bytes...
er.java:266)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.BinarySumLayer$Accumulator.lambda$accept$1(BinarySumLayer.java:324)
... 142 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 158 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy51.run(Unknown Source)
at com.simiacryptus.util.Util.lambda$runAllParallel$4(Util.java:399)
at com.simiacryptus.ref.wrappers.RefStream.lambda$forEach$25(RefStream.java:354)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at com.simiacryptus.ref.wrappers.StreamWrapper.forEach(StreamWrapper.java:129)
at com.simiacryptus.ref.wrappers.RefStream.forEach(RefStream.java:354)
at com.simiacryptus.util.Util.runAllParallel(Util.java:397)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:749)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:616)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.lambda$accept$2(ImgConcatLayer.java:381)
... 162 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 190 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.lang.ResourcePool.apply(ResourcePool.java:91)
at com.simiacryptus.mindseye.lang.cudnn.CudaSystem.run(CudaSystem.java:655)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$accept$3(SimpleConvolutionLayer.java:732)
... 194 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 198 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 6 more
Caused by: java.lang.IllegalStateException: Object com.simiacryptus.mindseye.lang.cudnn.CudaTensor (0 refs; 1 adds, 0 frees) apply current stack
at com.simiacryptus.ref.lang.ReferenceCountingBase.getStackTrace(ReferenceCountingBase.java:85)
at com.simiacryptus.ref.lang.ReferenceCountingBase.referenceReport(ReferenceCountingBase.java:325)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
... 10 more
Final threshold in iteration 0: 0.8831906026666665 (> 0.0) after 0.023s (< 30.000s)
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:57)
at com.simiacryptus.mindseye.eval.BatchedTrainable.measure(BatchedTrainable.java:66)
at com.simiacryptus.mindseye.opt.IterativeTrainer.measure(IterativeTrainer.java:143)
at com.simiacryptus.mindseye.opt.IterativeTrainer.run(IterativeTrainer.java:193)
at com.simiacryptus.mindseye.test.unit.TrainingTester.lambda$trainCjGD$892d3ca8$1(TrainingTester.java:461)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainCjGD(TrainingTester.java:452)
at com.simiacryptus.mindseye.test.unit.TrainingTester.runOpt(TrainingTester.java:658)
at com.simiacryptus.mindseye.test.unit.TrainingTester.train(TrainingTester.java:606)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainAll(TrainingTester.java:416)
at com.simiacryptus.mindseye.test.unit.TrainingTester.testCompleteLearning(TrainingTester.java:318)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:247)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:58)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambd
...skipping 5758 bytes...
om.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.accept(ImgConcatLayer.java:266)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.BinarySumLayer$Accumulator.lambda$accept$1(BinarySumLayer.java:324)
... 142 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 158 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy51.run(Unknown Source)
at com.simiacryptus.util.Util.lambda$runAllParallel$4(Util.java:399)
at com.simiacryptus.ref.wrappers.RefStream.lambda$forEach$25(RefStream.java:354)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at com.simiacryptus.ref.wrappers.StreamWrapper.forEach(StreamWrapper.java:129)
at com.simiacryptus.ref.wrappers.RefStream.forEach(RefStream.java:354)
at com.simiacryptus.util.Util.runAllParallel(Util.java:397)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:749)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:616)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.lambda$accept$2(ImgConcatLayer.java:381)
... 162 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 190 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.lang.ResourcePool.apply(ResourcePool.java:91)
at com.simiacryptus.mindseye.lang.cudnn.CudaSystem.run(CudaSystem.java:655)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$accept$3(SimpleConvolutionLayer.java:732)
... 194 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 198 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 6 more
Caused by: java.lang.IllegalStateException: Object com.simiacryptus.mindseye.lang.cudnn.CudaTensor (0 refs; 1 adds, 0 frees) apply current stack
at com.simiacryptus.ref.lang.ReferenceCountingBase.getStackTrace(ReferenceCountingBase.java:85)
at com.simiacryptus.ref.lang.ReferenceCountingBase.referenceReport(ReferenceCountingBase.java:325)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.simiacryptus.ref.lang.ReferenceCountingBase.addRef(ReferenceCountingBase.java:170)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.addRef(CudaTensor.java:266)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:245)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
... 10 more
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[0.048, -0.852, 1.912, -0.384, 1.616, 1.524, -1.72, -0.128, 1.208]
[0.08, 0.7, 1.764, 1.108, -1.028, 0.496, -0.608, -0.804, 0.028]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.08, -0.128, 0.7 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 1.451296, 0.829232, 0.7243839999999999, 1.1516, 0.5674239999999999, 0.07387200000000001 ] ]
]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.01 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 11415349374521
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:57)
at com.simiacryptus.mindseye.eval.BatchedTrainable.measure(BatchedTrainable.java:66)
at com.simiacryptus.mindseye.opt.IterativeTrainer.measure(IterativeTrainer.java:143)
at com.simiacryptus.mindseye.opt.IterativeTrainer.run(IterativeTrainer.java:185)
at com.simiacryptus.mindseye.test.unit.TrainingTester.lambda$trainLBFGS$892d3ca8$1(TrainingTester.java:519)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainLBFGS(TrainingTester.java:509)
at com.simiacryptus.mindseye.test.unit.TrainingTester.runOpt(TrainingTester.java:658)
at com.simiacryptus.mindseye.test.unit.TrainingTester.train(TrainingTester.java:606)
at com.simiacryptus.mindseye.test.unit.TrainingTester.trainAll(TrainingTester.java:418)
at com.simiacryptus.mindseye.test.unit.TrainingTester.testCompleteLearning(TrainingTester.java:318)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:247)
at com.simiacryptus.mindseye.test.unit.TrainingTester.test(TrainingTester.java:58)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lam
...skipping 4565 bytes...
$measure$3f677$1(BasicTrainable.java:127)
at com.simiacryptus.lang.TimedResult.time(TimedResult.java:54)
at com.simiacryptus.mindseye.eval.BasicTrainable.measure(BasicTrainable.java:127)
at com.simiacryptus.mindseye.eval.TrainableWrapper.measure(TrainableWrapper.java:70)
at com.simiacryptus.mindseye.eval.BatchedTrainable.lambda$measure$8766d393$1(BatchedTrainable.java:96)
... 89 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 138 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy54.accept(Unknown Source)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:581)
at com.simiacryptus.ref.wrappers.RefIntStream.forEach(RefIntStream.java:201)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.accept(ImgConcatLayer.java:302)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.accept(ImgConcatLayer.java:266)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.BinarySumLayer$Accumulator.lambda$accept$1(BinarySumLayer.java:324)
... 142 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 158 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy51.run(Unknown Source)
at com.simiacryptus.util.Util.lambda$runAllParallel$4(Util.java:399)
at com.simiacryptus.ref.wrappers.RefStream.lambda$forEach$25(RefStream.java:354)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at com.simiacryptus.ref.wrappers.StreamWrapper.forEach(StreamWrapper.java:129)
at com.simiacryptus.ref.wrappers.RefStream.forEach(RefStream.java:354)
at com.simiacryptus.util.Util.runAllParallel(Util.java:397)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:749)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.accept(SimpleConvolutionLayer.java:616)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accum(CountingResult.java:115)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:108)
at com.simiacryptus.mindseye.network.CountingResult$CountingAccumulator.accept(CountingResult.java:76)
at com.simiacryptus.mindseye.layers.cudnn.ImgConcatLayer$Accumulator.lambda$accept$2(ImgConcatLayer.java:381)
... 162 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 190 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.lang.ResourcePool.apply(ResourcePool.java:91)
at com.simiacryptus.mindseye.lang.cudnn.CudaSystem.run(CudaSystem.java:655)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$accept$3(SimpleConvolutionLayer.java:732)
... 194 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 198 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy56.apply(Unknown Source)
at com.simiacryptus.mindseye.lang.cudnn.CudnnHandle.lambda$call$3(CudnnHandle.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 6 more
Caused by: com.simiacryptus.mindseye.lang.ComponentException: Error in convolution [1, 1, 3] x [1, 1, 3, 3]
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:284)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.bck2(SimpleConvolutionLayer.java:349)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.access$100(SimpleConvolutionLayer.java:44)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer$Accumulator.lambda$null$2(SimpleConvolutionLayer.java:738)
... 10 more
Caused by: com.simiacryptus.ref.lang.LifecycleException: Lifecycle Exception: Object com.simiacryptus.mindseye.lang.cudnn.CudaTensor (0 refs; 1 adds, 0 frees)
at com.simiacryptus.ref.lang.ReferenceCountingBase.assertAlive(ReferenceCountingBase.java:188)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.getMemory(CudaTensor.java:90)
at com.simiacryptus.mindseye.lang.cudnn.CudaTensor.getMemory(CudaTensor.java:85)
at com.simiacryptus.mindseye.layers.cudnn.conv.SimpleConvolutionLayer.fwd(SimpleConvolutionLayer.java:252)
... 13 more
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[0.048, -0.852, 1.912, -0.384, 1.616, 1.524, -1.72, -0.128, 1.208]
[0.08, 0.7, 1.764, 1.108, -1.028, 0.496, -0.608, -0.804, 0.028]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.08, -0.128, 0.7 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 1.451296, 0.829232, 0.7243839999999999, 1.1516, 0.5674239999999999, 0.07387200000000001 ] ]
]
TrainingTester.java:432 executed in 0.00 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
No Data
TrainingTester.java:435 executed in 0.00 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
No Data
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }, "model":{ "LBFGS": { "type": "Converged", "value": 1.6049416203226965E-35 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 1.6049416203226965E-35 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }, "model":{ "LBFGS": { "type": "Converged", "value": 1.6049416203226965E-35 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 1.6049416203226965E-35 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }} | OK |
{
"result": "OK",
"performance": {
"execution_time": "9.806",
"gc_time": "0.898"
},
"created_on": 1586746037809,
"file_name": "trainingTest",
"report": {
"simpleName": "BandExpand",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.BandExpand",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": "NaN"
},
"CjGD": {
"type": "NonConverged",
"value": "NaN"
},
"GD": {
"type": "NonConverged",
"value": "NaN"
}
},
"model": {
"LBFGS": {
"type": "Converged",
"value": 1.6049416203226965E-35
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 1.6049416203226965E-35
}
},
"complete": {
"LBFGS": {
"type": "NonConverged",
"value": "NaN"
},
"CjGD": {
"type": "NonConverged",
"value": "NaN"
},
"GD": {
"type": "NonConverged",
"value": "NaN"
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/BandExpand/trainingTest/202004134717",
"id": "9f8b6276-5b9e-4171-8614-bba0a361fa98",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ConvolutionLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
"javaDoc": ""
}
}