Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
This is a network apply the following layout:
LayerTests.java:203 executed in 10.00 seconds (0.000 gc):
return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
.render(Format.PNG).toImage();
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:659)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.LayerTests.renderGraph(LayerTests.java:203)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:415)
at com.simiacryptus.mindseye.test.LayerTestBase.trainingTest(LayerTestBase.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(Unknown Source)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(Unknown Source)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambda$null$8bb7732f$1(MacroTestRunner.java:177)
at com.simiacryptus.aws.TendrilControl.lambda$null$0(TendrilControl.java:86)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy44.get(Unknown Source)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
... 71 more
Caused by: guru.nidi.graphviz.engine.GraphvizException: Engine took too long to respond, try setting a higher timout
at guru.nidi.graphviz.engine.GraphvizServerEngine.execute(GraphvizServerEngine.java:49)
at guru.nidi.graphviz.engine.Graphviz.execute(Graphviz.java:246)
at guru.nidi.graphviz.engine.Renderer.execute(Renderer.java:134)
at guru.nidi.graphviz.engine.Renderer.toImage(Renderer.java:105)
at com.simiacryptus.mindseye.test.unit.LayerTests.lambda$renderGraph$12c65ef7$1(LayerTests.java:205)
... 77 more
Using Seed 3018658400578148352
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[ 0.7, 0.08, -0.128 ]
[ 0.08, -0.128, 0.7 ]
[ 0.08, -0.128, 0.7 ]
[ 0.08, -0.128, 0.7 ]
[ -0.128, 0.08, 0.7 ]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.25 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3445937007542
Reset training subject: 3445952275108
Constructing line search parameters: GD
th(0)=0.2349000004955351;dx=-0.029195272125444385
New Minimum: 0.2349000004955351 > 0.1792666543520943
END: th(2.154434690031884)=0.1792666543520943; dx=-0.022960528663676837 evalInputDelta=0.055633346143440815
Fitness changed from 0.2349000004955351 to 0.1792666543520943
Iteration 1 complete. Error: 0.1792666543520943 Total: 0.0539; Orientation: 0.0028; Line Search: 0.0256
th(0)=0.1792666543520943;dx=-0.018083138971781293
New Minimum: 0.1792666543520943 > 0.10746680805384093
END: th(4.641588833612779)=0.10746680805384093; dx=-0.01333900673766253 evalInputDelta=0.07179984629825337
Fitness changed from 0.1792666543520943 to 0.10746680805384093
Iteration 2 complete. Error: 0.10746680805384093 Total: 0.0182; Orientation: 0.0011; Line Search: 0.0111
th(0)=0.10746680805384093;dx=-0.009848857057465469
New Minimum: 0.10746680805384093 > 0.034241606698123726
END: th(10.000000000000002)=0.034241606698123726; dx=-0.007591543560916324 evalInputDelta=0.0732252013557172
Fitness changed from 0.10746680805384093 to 0.034241606698123726
Iteration 3 complete. Error: 0.034241606698123726 Total: 0.0182; Orientation: 0.0010; Line Search: 0.0102
th(0)=0.034241606698123726;dx=-0.005914044558517259
New Minimum: 0.034241606698123726 > 0.0
WOLFE (weak): th(21.544346900318843)=0.0; dx=-0.00723867934089307 evalInputDelta=0.034241606698123726
WOLFE (weak): th(43.088693800637685)=0.0; dx=-0.00969666928586836 evalInputDelta=0.034241606698123726
WOLFE (weak): th(129.26608140191306)=0.0; dx=-0.019528629065769517 evalInputDelta=0.034241606698123726
WOLFE (weak): th(517.0643256076522)=0.0; dx=-0.06377244807532473 evalInputDelta=0.034241606698123726
WOLFE (weak): th(2585.321628038261)=0.0; dx=-0.2997394827929526 evalInputDelta=0.034241606698123726
WOLFE (weak): th(15511.929768229567)=0.0; dx=-1.7745334497781269 evalInputDelta=0.034241606698123726
WOLFE (weak): th(108583.50837760697)=0.0; dx=-12.393050012071377 evalInputDelta=0.034241606698123726
WOLFE (weak): th(868668.0670208557)=0.0; dx=-99.11093527079959 evalInputDelta=0.034241606698123726
Armijo: th(7818012.603187702)=0.0; dx=-891.9601719220292 evalInputDelta=0.034241606698123726
WOLFE (weak): th(4343340.335104278)=0.0; dx=-495.5355535964143 evalInputDelta=0.034241606698123726
Armijo: th(6080676.46914599)=0.0; dx=-693.7478627592217 evalInputDelta=0.034241606698123726
WOLFE (weak): th(5212008.402125134)=0.0; dx=-594.641708177818 evalInputDelta=0.034241606698123726
WOLFE (weak): th(5646342.435635562)=0.0; dx=-644.1947854685199 evalInputDelta=0.034241606698123726
Armijo: th(5863509.452390776)=0.0; dx=-668.9713241138709 evalInputDelta=0.034241606698123726
WOLFE (weak): th(5754925.944013169)=0.0; dx=-656.5830547911953 evalInputDelta=0.034241606698123726
Armijo: th(5809217.698201973)=0.0; dx=-662.7771894525332 evalInputDelta=0.034241606698123726
mu ~= nu (5754925.944013169): th(21.544346900318843)=0.0
Fitness changed from 0.034241606698123726 to 0.0
Iteration 4 complete. Error: 0.0 Total: 0.0780; Orientation: 0.0026; Line Search: 0.0705
th(0)=0.0;dx=-0.009458569013954626
Armijo: th(1.2457096111649979E7)=0.0; dx=-2661.1266061273864 evalInputDelta=0.0
Armijo: th(6228548.055824989)=0.0; dx=-1330.5680323482 evalInputDelta=0.0
Armijo: th(2076182.6852749966)=0.0; dx=-443.5289831620761 evalInputDelta=0.0
Armijo: th(519045.67131874914)=0.0; dx=-110.88933971727947 evalInputDelta=0.0
Armijo: th(103809.13426374983)=0.0; dx=-22.185434798667057 evalInputDelta=0.0
Armijo: th(17301.52237729164)=0.0; dx=-3.7054546072894725 evalInputDelta=0.0
Armijo: th(2471.646053898806)=0.0; dx=-0.5374580030533143 evalInputDelta=0.0
Armijo: th(308.9557567373507)=0.0; dx=-0.0754584982688746 evalInputDelta=0.0
Armijo: th(34.32841741526119)=0.0; dx=-0.01679189448672351 evalInputDelta=0.0
Armijo: th(3.4328417415261194)=0.0; dx=-0.010191901561231512 evalInputDelta=0.0
Armijo: th(0.31207652195691993)=0.0; dx=-0.009525235609161615 evalInputDelta=0.0
Armijo: th(0.026006376829743327)=0.0; dx=-0.009464124563555207 evalInputDelta=0.0
Armijo: th(0.002000490525364871)=0.0; dx=-0.0094589963639239 evalInputDelta=0.0
Armijo: th(1.4289218038320508E-4)=0.0; dx=-0.00945859953895243 evalInputDelta=0.0
Armijo: th(9.526145358880338E-6)=0.0; dx=-0.009458571048954478 evalInputDelta=0.0
Armijo: th(5.953840849300211E-7)=0.0; dx=-0.009458569141142115 evalInputDelta=0.0
Armijo: th(3.502259323117771E-8)=0.0; dx=-0.009458569021436242 evalInputDelta=0.0
Armijo: th(1.9456996239543175E-9)=0.0; dx=-0.009458569014370269 evalInputDelta=0.0
Armijo: th(1.024052433660167E-10)=0.0; dx=-0.0094585690139765 evalInputDelta=0.0
Armijo: th(5.120262168300835E-12)=0.0; dx=-0.009458569013955719 evalInputDelta=0.0
Armijo: th(2.4382200801432547E-13)=0.0; dx=-0.009458569013954678 evalInputDelta=0.0
Armijo: th(1.1082818546105704E-14)=0.0; dx=-0.009458569013954627 evalInputDelta=0.0
MIN ALPHA (4.818616759176393E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0761; Orientation: 0.0008; Line Search: 0.0727
Iteration 5 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 5
Final threshold in iteration 5: 0.0 (> 0.0) after 0.245s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.09 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3446188730202
Reset training subject: 3446191229498
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=0.2349000004955351}, derivative=-0.029195272125444385}
New Minimum: 0.2349000004955351 > 0.23490000049261556
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=0.23490000049261556}, derivative=-0.02919527212507404}, evalInputDelta = -2.919553487856774E-12
New Minimum: 0.23490000049261556 > 0.23490000047509843
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=0.23490000047509843}, derivative=-0.029195272122851966}, evalInputDelta = -2.043668012596811E-11
New Minimum: 0.23490000047509843 > 0.23490000035247824
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=0.23490000035247824}, derivative=-0.02919527210729745}, evalInputDelta = -1.4305687190407923E-10
New Minimum: 0.23490000035247824 > 0.23489999949413726
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=0.23489999949413726}, derivative=-0.029195271998415838}, evalInputDelta = -1.001397853528374E-9
New Minimum: 0.23489999949413726 > 0.23489999348575036
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=0.23489999348575036}, derivative=-0.029195271236244585}, evalInputDelta = -7.0097847526540136E-9
New Minimum: 0.23489999348575036 > 0.23489995142704645
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=0.23489995142704645}, derivative=-0.029195265901046933}, evalInputDelta = -4.906848866115254E-8
New Minimum: 0.23489995142704645 > 0.23489965701633436
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=0.23489965701633436}, derivative=-0.02919522855471955}, evalInputDelta = -3.4347920074839777E-7
New Minimum: 0.23489965701633436 > 0.23489759615189465
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=0.23489759615189465}, derivative=-0.0291949671331797}, evalInputDelta = -2.4043436404608265E-6
New Minimum: 0.23489759615189465 > 0.23488317061749334
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=0.23488317061749334}, derivative=-0.02919313731723018}, evalInputDelta = -1.682987804177416E-5
New Minimum: 0.23488317061749334 > 0.23478221718498582
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=0.23478221718498582}, derivative=-0.029180335208687055}, evalInputDelta = -1.1778331054929092E-4
New Minimum: 0.23478221718498582 > 0.23407678023465434
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=0.23407678023465434}, derivative=-0.029091042791410567}, evalInputDelta = -8.232202608807748E-4
New Minimum: 0.23407678023465434 > 0.22919831931654197
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=0.22919831931654197}, derivative=-0.028481386578113832}, evalInputDelta = -0.00570168117899314
New Minimum: 0.22919831931654197 > 0.19766184504662915
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=0.19766184504662915}, derivative=-0.024851146759191638}, evalInputDelta = -0.03723815544890596
New Minimum: 0.19766184504662915 > 0.05195173203950069
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=0.05195173203950069}, derivative=-0.013503929267508392}, evalInputDelta = -0.18294826845603443
New Minimum: 0.05195173203950069 > 0.0
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.04093851704179548}, evalInputDelta = -0.2349000004955351
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.26118079649150167}, evalInputDelta = -0.2349000004955351
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.8028767526394454}, evalInputDelta = -0.2349000004955351
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-12.594748445675052}, evalInputDelta = -0.2349000004955351
F(162841.3597910449) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-88.13785029692427}, evalInputDelta = -0.2349000004955351
F(1139889.5185373144) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-616.939563255669}, evalInputDelta = -0.2349000004955351
F(7979226.6297612) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-4318.55155396688}, evalInputDelta = -0.2349000004955351
F(5.58545864083284E7) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-30229.835488945362}, evalInputDelta = -0.2349000004955351
F(3.909821048582988E8) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-211608.82303379476}, evalInputDelta = -0.2349000004955351
F(2.7368747340080914E9) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1481261.7358477404}, evalInputDelta = -0.2349000004955351
F(1.915812313805664E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.036883212554536E7}, evalInputDelta = -0.2349000004955351
0.0 <= 0.2349000004955351
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-5412237.960211997}, evalInputDelta = -0.2349000004955351
Right bracket at 1.0E10
Converged to right
Fitness changed from 0.2349000004955351 to 0.0
Iteration 1 complete. Error: 0.0 Total: 0.0859; Orientation: 0.0007; Line Search: 0.0771
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-0.0638871551107433}
F(1.0E10) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=-1.451602128035324E7}, evalInputDelta = 0.0
0.0 <= 0.0
Converged to right
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0074; Orientation: 0.0007; Line Search: 0.0048
Iteration 2 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 2
Final threshold in iteration 2: 0.0 (> 0.0) after 0.094s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.17 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 3446288129723
Reset training subject: 3446290433031
Adding measurement 3e76aec1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 0.2349000004955351 < 0.2349000004955351. Total: 1
th(0)=0.2349000004955351;dx=-0.029195272125444385
Adding measurement 2996538c to history. Total: 1
New Minimum: 0.2349000004955351 > 0.1792666543520943
END: th(2.154434690031884)=0.1792666543520943; dx=-0.022960528663676837 evalInputDelta=0.055633346143440815
Fitness changed from 0.2349000004955351 to 0.1792666543520943
Iteration 1 complete. Error: 0.1792666543520943 Total: 0.0159; Orientation: 0.0033; Line Search: 0.0068
Non-optimal measurement 0.1792666543520943 < 0.1792666543520943. Total: 2
LBFGS Accumulation History: 2 points
Non-optimal measurement 0.1792666543520943 < 0.1792666543520943. Total: 2
th(0)=0.1792666543520943;dx=-0.018083138971781293
Adding measurement 55ce57a8 to history. Total: 2
New Minimum: 0.1792666543520943 > 0.10746680805384093
END: th(4.641588833612779)=0.10746680805384093; dx=-0.01333900673766253 evalInputDelta=0.07179984629825337
Fitness changed from 0.1792666543520943 to 0.10746680805384093
Iteration 2 complete. Error: 0.10746680805384093 Total: 0.0127; Orientation: 0.0028; Line Search: 0.0078
Non-optimal measurement 0.10746680805384093 < 0.10746680805384093. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 0.10746680805384093 < 0.10746680805384093. Total: 3
th(0)=0.10746680805384093;dx=-0.009848857057465469
Adding measurement 19b38ac3 to history. Total: 3
New Minimum: 0.10746680805384093 > 0.034241606698123726
END: th(10.000000000000002)=0.034241606698123726; dx=-0.007591543560916324 evalInputDelta=0.0732252013557172
Fitness changed from 0.10746680805384093 to 0.034241606698123726
Iteration 3 complete. Error: 0.034241606698123726 Total: 0.0082; Orientation: 0.0012; Line Search: 0.0057
Non-optimal measurement 0.034241606698123726 < 0.034241606698123726. Total: 4
Rejected: LBFGS Orientation magnitude: 8.006e-01, gradient 7.690e-02, dot -0.991; [78e86100-e379-49bd-8693-3383cc5e0737 = 1.000/1.000e+00, 69aee432-4c91-42b9-bad8-c2ff9e7b28b1 = 1.000/1.000e+00, 1f72138b-263d-4c47-8a3c-1d943d75c10e = 1.000/1.000e+00, efb1b3a1-95c3-4f30-8d93-18cd7444e8ad = 1.000/1.000e+00, 585e6e41-ad04-4479-a06d-71a1bedd56e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.034241606698123726, 0.10746680805384093, 0.1792666543520943, 0.2349000004955351
LBFGS Accumulation History: 3 points
Removed measurement 19b38ac3 to history. Total: 3
Adding measurement 45de68f1 to history. Total: 3
th(0)=0.034241606698123726;dx=-0.005914044558517259
Adding measurement 75d58e35 to history. Total: 4
New Minimum: 0.034241606698123726 > 0.0
WOLFE (weak): th(21.544346900318843)=0.0; dx=-0.00723867934089307 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(43.088693800637685)=0.0; dx=-0.00969666928586836 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(129.26608140191306)=0.0; dx=-0.01952862906576952 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(517.0643256076522)=0.0; dx=-0.06377244807532474 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(2585.321628038261)=0.0; dx=-0.2997394827929526 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(15511.929768229567)=0.0; dx=-1.7745334497781269 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(108583.50837760697)=0.0; dx=-12.393050012071377 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(868668.0670208557)=0.0; dx=-99.11093527079959 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(7818012.603187702)=0.0; dx=-891.9601719220292 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(4343340.335104278)=0.0; dx=-495.53555359641433 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(6080676.46914599)=0.0; dx=-693.7478627592217 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(5212008.402125134)=0.0; dx=-594.641708177818 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(5646342.435635562)=0.0; dx=-644.1947854685199 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5863509.452390776)=0.0; dx=-668.9713241138709 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
WOLFE (weak): th(5754925.944013169)=0.0; dx=-656.5830547911953 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
Armijo: th(5809217.698201973)=0.0; dx=-662.7771894525332 evalInputDelta=0.034241606698123726
Non-optimal measurement 0.0 < 0.0. Total: 5
mu ~= nu (5754925.944013169): th(21.544346900318843)=0.0
Fitness changed from 0.034241606698123726 to 0.0
Iteration 4 complete. Error: 0.0 Total: 0.0658; Orientation: 0.0236; Line Search: 0.0399
Non-optimal measurement 0.0 < 0.0. Total: 5
Rejected: LBFGS Orientation magnitude: 9.567e-01, gradient 9.726e-02, dot -0.980; [1f72138b-263d-4c47-8a3c-1d943d75c10e = 1.000/1.000e+00, 78e86100-e379-49bd-8693-3383cc5e0737 = 1.000/1.000e+00, 585e6e41-ad04-4479-a06d-71a1bedd56e5 = 1.000/1.000e+00, efb1b3a1-95c3-4f30-8d93-18cd7444e8ad = 1.000/1.000e+00, 69aee432-4c91-42b9-bad8-c2ff9e7b28b1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.034241606698123726, 0.10746680805384093, 0.1792666543520943, 0.2349000004955351
Rejected: LBFGS Orientation magnitude: 1.655e+00, gradient 9.726e-02, dot -0.987; [69aee432-4c91-42b9-bad8-c2ff9e7b28b1 = 1.000/1.000e+00, efb1b3a1-95c3-4f30-8d93-18cd7444e8ad = 1.000/1.000e+00, 78e86100-e379-49bd-8693-3383cc5e0737 = 1.000/1.000e+00, 1f72138b-263d-4c47-8a3c-1d943d75c10e = 1.000/1.000e+00, 585e6e41-ad04-4479-a06d-71a1bedd56e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0, 0.034241606698123726, 0.10746680805384093, 0.1792666543520943
LBFGS Accumulation History: 3 points
Removed measurement 75d58e35 to history. Total: 4
Removed measurement 45de68f1 to history. Total: 3
Adding measurement 63362e67 to history. Total: 3
th(0)=0.0;dx=-0.009458569013954626
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.2457096111649979E7)=0.0; dx=-2661.1266061273864 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(6228548.055824989)=0.0; dx=-1330.5680323482 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2076182.6852749966)=0.0; dx=-443.5289831620761 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(519045.67131874914)=0.0; dx=-110.88933971727947 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(103809.13426374983)=0.0; dx=-22.185434798667057 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(17301.52237729164)=0.0; dx=-3.7054546072894725 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2471.646053898806)=0.0; dx=-0.5374580030533145 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(308.9557567373507)=0.0; dx=-0.0754584982688746 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(34.32841741526119)=0.0; dx=-0.01679189448672351 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.4328417415261194)=0.0; dx=-0.010191901561231512 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.31207652195691993)=0.0; dx=-0.009525235609161615 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.026006376829743327)=0.0; dx=-0.009464124563555207 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(0.002000490525364871)=0.0; dx=-0.0094589963639239 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.4289218038320508E-4)=0.0; dx=-0.009458599538952431 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(9.526145358880338E-6)=0.0; dx=-0.009458571048954478 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(5.953840849300211E-7)=0.0; dx=-0.009458569141142115 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(3.502259323117771E-8)=0.0; dx=-0.009458569021436242 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.9456996239543175E-9)=0.0; dx=-0.009458569014370269 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.024052433660167E-10)=0.0; dx=-0.0094585690139765 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(5.120262168300835E-12)=0.0; dx=-0.009458569013955717 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(2.4382200801432547E-13)=0.0; dx=-0.009458569013954678 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
Armijo: th(1.1082818546105704E-14)=0.0; dx=-0.009458569013954627 evalInputDelta=0.0
Non-optimal measurement 0.0 < 0.0. Total: 4
MIN ALPHA (4.818616759176393E-16): th(0.0)=0.0
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0702; Orientation: 0.0261; Line Search: 0.0425
Iteration 5 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 5
Final threshold in iteration 5: 0.0 (> 0.0) after 0.173s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.09 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -1.4654458653804507], [4.0, -0.7465004866658735]; valueStats=DoubleSummaryStatistics{count=6, sum=0.641950, min=0.034242, average=0.106992, max=0.179267}
Plotting 4 points for GD
Only 1 points for CjGD
Plotting 4 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -1.4654458653804507], [0.114, -0.7465004866658735]; valueStats=DoubleSummaryStatistics{count=6, sum=0.641950, min=0.034242, average=0.106992, max=0.179267}
Plotting 4 points for GD
Only 1 points for CjGD
Plotting 4 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "11.190",
"gc_time": "0.286"
},
"created_on": 1586738023733,
"file_name": "trainingTest",
"report": {
"simpleName": "Normal",
"canonicalName": "com.simiacryptus.mindseye.layers.java.TargetValueLayerTest.Normal",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/TargetValueLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/TargetValueLayer/Normal/trainingTest/202004133343",
"id": "824c5a8e-e6d2-4488-b5d1-823da1b2869e",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "TargetValueLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.TargetValueLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/TargetValueLayer.java",
"javaDoc": ""
}
}