Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
This is a network apply the following layout:
LayerTests.java:203 executed in 0.07 seconds (0.000 gc):
return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
.render(Format.PNG).toImage();
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine6534774723062079087/dotfile.dot -ooutfile.svg]
Returns
Using Seed 2370656357698341888
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.01 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.552, 1.292, -0.964 ], [ -0.556, 0.7, 1.04 ], [ 1.268, 0.776, -0.452 ], [ 1.108, 0.124, -1.044 ], [ -0.492, 0.236, 0.524 ], [ -1.052, 0.752, -0.46 ], [ -1.82, -0.204, 0.34 ], [ 1.176, 0.476, 1.792 ], ... ],
[ [ 0.652, -0.864, -1.6 ], [ -0.344, 1.5, -1.072 ], [ 0.176, 0.176, -1.5 ], [ 0.476, 0.692, 0.056 ], [ 0.532, 0.104, -1.584 ], [ 0.084, 1.668, 1.964 ], [ -0.488, 0.944, -1.932 ], [ -1.508, -1.696, 0.124 ], ... ],
[ [ 1.824, 0.944, 1.184 ], [ 1.02, 0.604, 0.708 ], [ 0.292, 0.828, -1.524 ], [ -1.06, 1.736, -1.508 ], [ -0.264, 0.936, -0.916 ], [ 0.952, 1.88, 1.86 ], [ -0.688, -0.8, -0.164 ], [ 0.632, -1.152, -1.968 ], ... ],
[ [ -0.008, -0.944, 0.896 ], [ 0.696, -1.044, -1.004 ], [ 1.792, 1.424, -0.12 ], [ 1.964, 1.572, 1.532 ], [ 1.156, 1.024, 0.5 ], [ 1.104, -1.212, -0.336 ], [ 1.328, -1.356, -1.444 ], [ -0.672, -1.16, 1.156 ], ... ],
[ [ -1.596, -0.892, -0.8 ], [ 0.54, 0.036, 1.692 ], [ 1.604, 1.864, -0.936 ], [ 0.412, -1.732, 0.532 ], [ 1.392, 1.616, 0.52 ], [ -0.364, -0.812, -0.624 ], [ 0.408, -1.272, 1.992 ], [ 1.304, 1.584, -0.64 ], ... ],
[ [ 0.6, 0.624, -0.312 ], [ 1.68, 0.032, 0.392 ], [ 0.632, -0.604, 0.824 ], [ 0.184, 1.752, 0.728 ], [ 0.2, 0.968, -0.9 ], [ -1.252, -1.676, -1.348 ], [ -0.424, -1.9, -0.844 ], [ 1.904, 1.484, -1.932 ], ... ],
[ [ 1.968, -0.932, 0.616 ], [ 1.72, 0.776, 0.884 ], [ 1.336, 0.708, -1.704 ], [ -1.348, -1.02, 0.84 ], [ -1.412, 0.02, 0.96 ], [ 0.336, 1.452, 0.144 ], [ -1.38, -1.032, 0.008 ], [ -0.212, -1.996, -1.964 ], ... ],
[ [ -0.572, 1.68, -1.5 ], [ -1.648, 0.768, 0.948 ], [ -1.068, 1.88, 0.956 ], [ -1.156, -1.0, 1.956 ], [ 0.18, 1.184, 0.672 ], [ -0.704, -1.504, 1.248 ], [ 1.804, -1.724, 1.38 ], [ 0.708, 1.316, 1.972 ], ... ],
...
]
[
[ [ -0.96, 1.416, -1.876 ], [ 0.092, -1.32, 0.58 ], [ -0.64, 1.276, -1.02 ], [ 0.52, -1.956, 1.044 ], [ 0.32, -1.832, -1.968 ], [ 0.376, -1.528, -1.448 ], [ 0.596, -0.516, -0.592 ], [ -0.26, 1.284, -0.216 ], ... ],
[ [ 0.284, -0.016, 1.572 ], [ 0.548, 0.592, 0.64 ], [ 0.964, 0.156, -1.472 ], [ -0.276, -0.612, -1.904 ], [ -1.236, -1.088, -1.024 ], [ -0.824, 1.048, -0.36 ], [ 1.148, 0.488, -1.792 ], [ -1.776, -0.22, 0.36 ], ... ],
[ [ 0.008, 1.744, 0.928 ], [ -0.088, -0.612, -0.916 ], [ 1.04, -0.468, 1.884 ], [ 0.844, -0.684, -1.988 ], [ 1.724, -1.292, -0.172 ], [ -1.392, -0.488, -1.624 ], [ -0.048, 0.516, -1.08 ], [ -1.204, 0.844, 0.22 ], ... ],
[ [ 0.424, 1.744, 0.144 ], [ 1.184, 1.956, -1.872 ], [ 0.82, 1.316, 0.988 ], [ 0.88, 0.148, 1.94 ], [ 1.936, -0.26, 0.88 ], [ 1.94, -0.116, 0.364 ], [ 1.104, 0.588, 1.22 ], [ 0.58, 0.792, -1.74 ], ... ],
[ [ -1.672, 0.68, 1.876 ], [ 0.784, -0.808, 1.444 ], [ 1.196, 1.576, 1.352 ], [ 1.1, -1.8, -1.072 ], [ -1.94, 0.672, 0.76 ], [ 1.128, -0.552, -1.524 ], [ -1.232, -1.888, -0.736 ], [ 0.2, -0.336, 1.58 ], ... ],
[ [ 0.784, -1.68, 1.796 ], [ -1.408, 1.06, 0.932 ], [ 0.036, 1.12, -1.38 ], [ 0.712, -0.888, -0.792 ], [ 1.6, 1.92, 0.968 ], [ -0.736, -1.628, 0.816 ], [ -1.74, 0.708, -0.196 ], [ 0.64, 1.56, 1.788 ], ... ],
[ [ -1.736, 0.796, 0.348 ], [ 1.32, -1.272, -1.1 ], [ -0.6, -1.836, 0.216 ], [ -1.796, 0.16, 0.164 ], [ -0.328, -0.38, -0.628 ], [ -0.744, 0.596, -0.612 ], [ 0.996, 1.048, -1.448 ], [ 0.124, -0.168, -1.556 ], ... ],
[ [ 1.632, -0.856, 0.22 ], [ -0.64, 1.88, 0.924 ], [ 1.964, -0.444, -0.724 ], [ 0.556, 0.664, 0.32 ], [ -0.732, 1.156, 1.876 ], [ -0.468, -0.136, -0.904 ], [ 0.552, -0.336, 0.316 ], [ -0.744, -0.3, 1.98 ], ... ],
...
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 30.35 seconds (0.895 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13469868351396
Reset training subject: 13470199774923
Constructing line search parameters: GD
th(0)=94.4107808179455;dx=-0.012035856170180149
New Minimum: 94.4107808179455 > 94.38485247623717
WOLFE (weak): th(2.154434690031884)=94.38485247623717; dx=-0.01203388409931757 evalInputDelta=0.025928341708322478
New Minimum: 94.38485247623717 > 94.35892838322671
WOLFE (weak): th(4.308869380063768)=94.35892838322671; dx=-0.012031912028454994 evalInputDelta=0.05185243471878209
New Minimum: 94.35892838322671 > 94.25527449816366
WOLFE (weak): th(12.926608140191302)=94.25527449816366; dx=-0.012024023745004678 evalInputDelta=0.1555063197818356
New Minimum: 94.25527449816366 > 93.78967325755974
WOLFE (weak): th(51.70643256076521)=93.78967325755974; dx=-0.01198852646947826 evalInputDelta=0.6211075603857523
New Minimum: 93.78967325755974 > 91.32971551579132
WOLFE (weak): th(258.53216280382605)=91.32971551579132; dx=-0.011799207666670693 evalInputDelta=3.0810653021541725
New Minimum: 91.32971551579132 > 76.84210774656941
END: th(1551.1929768229563)=76.84210774656941; dx=-0.010615965149123403 evalInputDelta=17.568673071376082
Fitness changed from 94.4107808179455 to 76.84210774656941
Iteration 1 complete. Error: 76.84210774656941 Total: 2.1996; Orientation: 0.0840; Line Search: 1.6231
th(0)=76.84210774656941;dx=-0.009378147201831128
New Minimum: 76.84210774656941 > 49.38985895885882
END: th(3341.943960201201)=49.38985895885882; dx=-0.007050763105843709 evalInputDelta=27.452248787710595
Fitness changed from 76.84210774656941 to 49.38985895885882
Iteration 2 complete. Error: 49.38985895885882 Total: 0.4073; Orientation: 0.0672; Line Search: 0.2656
th(0)=49.38985895885882;dx=-0.005356624771513699
New Minimum: 49.38985895885882 > 20.460720238237112
END: th(7200.000000000001)=20.460720238237112; dx=-0.00267924709532566 evalInputDelta=28.929138720621705
Fitness changed from 49.38985895885882 to 20.460720238237112
Iteration 3 complete. Error: 20.460720238237112 Total: 0.4211; Orientation: 0.0685; Line Search: 0.2746
th(0)=20.460720238237112;dx=-0.0015043792851828848
New Minimum: 20.460720238237112 > 6.708659089542602
END: th(15511.929768229566)=6.708659089542602; dx=-2.687155333432413E-4 evalInputDelta=13.75206114869451
Fitness changed from 20.460720238237112 to 6.708659089542602
Iteration 4 complete. Error: 6.708659089542602 Total: 0.3975; Orientation: 0.0678; Line Search: 0.2563
th(0)=6.708659089542602;dx=-2.6135697596929553E-4
New Minimum: 6.708659089542602 > 3.3268686456608094
WOLF (strong): th(33419.43960201202)=3.3268686456608094; dx=5.897234689381631E-5 evalInputDelta=3.3817904438817923
END: th(16709.71980100601)=3.679610560329558; dx=-1.0119231453773963E-4 evalInputDelta=3.0290485292130436
Fitness changed from 6.708659089542602 to 3.3268686456608094
Iteration 5 complete. Error: 3.3268686456608094 Total: 0.5429; Orientation: 0.0973; Line Search: 0.3731
th(0)=3.3268686456608094;dx=-2.4259627374826102E-4
Armijo: th(36000.000000000015)=7.636271627881217; dx=4.8200755053828354E-4 evalInputDelta=-4.309402982220408
New Minimum: 3.3268686456608094 > 2.220852927481561
WOLF (strong): th(18000.000000000007)=2.220852927481561; dx=1.1970563839501129E-4 evalInputDelta=1.1060157181792483
END: th(6000.000000000003)=2.2335929153145155; dx=-1.2182896970050358E-4 evalInputDelta=1.0932757303462939
Fitness changed from 3.3268686456608094 to 2.220852927481561
Iteration 6 complete. Error: 2.220852927481561 Total: 0.7242; Orientation: 0.0674; Line Search: 0.5837
th(0)=2.220852927481561;dx=-1.2801619441207764E-4
New Minimum: 2.220852927481561 > 1.4757099575949835
WOLF (strong): th(12926.608140191309)=1.4757099575949835; dx=1.272795146305761E-5 evalInputDelta=0.7451429698865777
END: th(6463.304070095654)=1.6208633898190477; dx=-5.764412147451001E-5 evalInputDelta=0.5999895376625135
Fitness changed from 2.220852927481561 to 1.4757099575949835
Iteration 7 complete. Error: 1.4757099575949835 Total: 0.5180; Orientation: 0.0688; Line Search: 0.3740
th(0)=1.4757099575949835;dx=-2.4363733695560907E-5
New Minimum: 1.4757099575949835 > 1.2427168365477947
END: th(13924.766500838345)=1.2427168365477947; dx=-9.100830472629824E-6 evalInputDelta=0.23299312104718872
Fitness changed from 1.4757099575949835 to 1.2427168365477947
Iteration 8 complete. Error: 1.2427168365477947 Total: 0.3925; Orientation: 0.0683; Line Search: 0.2503
th(0)=1.2427168365477947;dx=-1.3429020323114232E-5
New Minimum: 1.2427168365477947 > 0.967403917530586
END: th(30000.00000000002)=0.967403917530586; dx=-4.9251742780330105E-6 evalInputDelta=0.2753129190172088
Fitness changed from 1.2427168365477947 to 0.967403917530586
Iteration 9 complete. Error: 0.967403917530586 Total: 0.4219; Orientation: 0.0697; Line Search: 0.2792
th(0)=0.967403917530586;dx=-1.533761346697027E-5
Armijo: th(64633.04070095655)=1.90703815575664; dx=4.4413585385838934E-5 evalInputDelta=-0.9396342382260541
New Minimum: 0.967403917530586 > 0.9544833282205422
WOLF (strong): th(32316.520350478277)=0.9544833282205422; dx=1.453798595943433E-5 evalInputDelta=0.012920589310043806
New Minimum: 0.9544833282205422 > 0.8558220081110001
END: th(10772.173450159426)=0.8558220081110001; dx=-5.379080324835405E-6 evalInputDelta=0.1115819094195859
Fitness changed from 0.967403917530586 to 0.8558220081110001
Iteration 10 complete. Error: 0.8558220081110001 Total: 0.6811; Orientation: 0.0704; Line Search: 0.5159
th(0)=0.8558220081110001;dx=-6.236196980031218E-6
New Minimum: 0.8558220081110001 > 0.7255054271447039
END: th(23207.94416806391)=0.7255054271447039; dx=-4.994145528774258E-6 evalInputDelta=0.13031658096629617
Fitness changed from 0.8558220081110001 to 0.7255054271447039
Iteration 11 complete. Error: 0.7255054271447039 Total: 0.4364; Orientation: 0.1029; Line Search: 0.2594
th(0)=0.7255054271447039;dx=-4.935813057723611E-6
New Minimum: 0.7255054271447039 > 0.5573207403842908
END: th(50000.00000000004)=0.5573207403842908; dx=-1.7915744126929096E-6 evalInputDelta=0.1681846867604131
Fitness changed from 0.7255054271447039 to 0.5573207403842908
Iteration 12 complete. Error: 0.5573207403842908 Total: 0.4058; Orientation: 0.0692; Line Search: 0.2634
th(0)=0.5573207403842908;dx=-9.630416847255714E-6
Armijo: th(107721.73450159427)=3.9389905999949946; dx=7.24157010845914E-5 evalInputDelta=-3.381669859610704
Armijo: th(53860.867250797135)=1.1433869035970265; dx=3.139264211866784E-5 evalInputDelta=-0.5860661632127357
New Minimum: 0.5573207403842908 > 0.5071719577679548
WOLF (strong): th(17953.62241693238)=0.5071719577679548; dx=4.043936141385473E-6 evalInputDelta=0.05014878261633604
END: th(4488.405604233095)=0.5217675287595274; dx=-6.211828600095417E-6 evalInputDelta=0.03555321162476344
Fitness changed from 0.5573207403842908 to 0.5071719577679548
Iteration 13 complete. Error: 0.5071719577679548 Total: 0.8482; Orientation: 0.0687; Line Search: 0.7060
th(0)=0.5071719577679548;dx=-8.35060273291969E-6
New Minimum: 0.5071719577679548 > 0.45735256902379484
END: th(9669.976736693297)=0.45735256902379484; dx=-1.953328724354098E-6 evalInputDelta=0.04981938874415992
Fitness changed from 0.5071719577679548 to 0.45735256902379484
Iteration 14 complete. Error: 0.45735256902379484 Total: 0.4058; Orientation: 0.0703; Line Search: 0.2591
th(0)=0.45735256902379484;dx=-2.1426344936577763E-6
New Minimum: 0.45735256902379484 > 0.4147012159854675
WOLFE (weak): th(20833.33333333335)=0.4147012159854675; dx=-1.951895398021641E-6 evalInputDelta=0.042651353038327344
New Minimum: 0.4147012159854675 > 0.3760235941062263
END: th(41666.6666666667)=0.3760235941062263; dx=-1.7611563023855074E-6 evalInputDelta=0.08132897491756852
Fitness changed from 0.45735256902379484 to 0.3760235941062263
Iteration 15 complete. Error: 0.3760235941062263 Total: 0.5232; Orientation: 0.0687; Line Search: 0.3806
th(0)=0.3760235941062263;dx=-1.7260363640667897E-6
New Minimum: 0.3760235941062263 > 0.3159987855454362
WOLF (strong): th(89768.1120846619)=0.3159987855454362; dx=3.8870605452035126E-7 evalInputDelta=0.06002480856079012
END: th(44884.05604233095)=0.3222816355180913; dx=-6.686651547732194E-7 evalInputDelta=0.05374195858813502
Fitness changed from 0.3760235941062263 to 0.3159987855454362
Iter
...skipping 11218 bytes...
466986575386938E-6 evalInputDelta=-0.02407089081203201
Armijo: th(30360.52165935911)=0.07689769997881837; dx=3.964176090599515E-7 evalInputDelta=-6.45983394543248E-4
New Minimum: 0.07625171658427513 > 0.07393605282435115
END: th(10120.173886453036)=0.07393605282435115; dx=-1.0376975659254338E-7 evalInputDelta=0.0023156637599239804
Fitness changed from 0.07625171658427513 to 0.07393605282435115
Iteration 38 complete. Error: 0.07393605282435115 Total: 0.6768; Orientation: 0.0665; Line Search: 0.5322
th(0)=0.07393605282435115;dx=-1.2820797088758807E-7
New Minimum: 0.07393605282435115 > 0.07124459236380978
WOLFE (weak): th(21803.25369012921)=0.07124459236380978; dx=-1.1867815893438848E-7 evalInputDelta=0.0026914604605413617
New Minimum: 0.07124459236380978 > 0.06876091281090324
END: th(43606.50738025842)=0.06876091281090324; dx=-1.0914834698118889E-7 evalInputDelta=0.005175140013447904
Fitness changed from 0.07393605282435115 to 0.06876091281090324
Iteration 39 complete. Error: 0.06876091281090324 Total: 0.5452; Orientation: 0.0666; Line Search: 0.4011
th(0)=0.06876091281090324;dx=-1.4770656484926347E-7
Armijo: th(93947.37221116011)=0.07271974384387182; dx=2.319841968850064E-7 evalInputDelta=-0.003958831032968582
New Minimum: 0.06876091281090324 > 0.066281459662664
WOLF (strong): th(46973.686105580055)=0.066281459662664; dx=4.213881601787147E-8 evalInputDelta=0.002479453148239247
END: th(15657.895368526684)=0.06694356872488494; dx=-8.442477122688518E-8 evalInputDelta=0.001817344086018302
Fitness changed from 0.06876091281090324 to 0.066281459662664
Iteration 40 complete. Error: 0.066281459662664 Total: 0.7762; Orientation: 0.0674; Line Search: 0.6304
th(0)=0.066281459662664;dx=-7.10788592368022E-7
Armijo: th(33733.91295484346)=0.08043930321681801; dx=1.5501720088786325E-6 evalInputDelta=-0.014157843554154018
New Minimum: 0.066281459662664 > 0.06382650042514285
WOLF (strong): th(16866.95647742173)=0.06382650042514285; dx=4.196917082553053E-7 evalInputDelta=0.002454959237521148
New Minimum: 0.06382650042514285 > 0.06334449969135736
END: th(5622.318825807243)=0.06334449969135736; dx=-3.339618254935795E-7 evalInputDelta=0.0029369599713066352
Fitness changed from 0.066281459662664 to 0.06334449969135736
Iteration 41 complete. Error: 0.06334449969135736 Total: 0.7298; Orientation: 0.0750; Line Search: 0.5718
th(0)=0.06334449969135736;dx=-1.8861882037123306E-7
New Minimum: 0.06334449969135736 > 0.061794565572465245
END: th(12112.918716738453)=0.061794565572465245; dx=-6.729540726249267E-8 evalInputDelta=0.0015499341188921142
Fitness changed from 0.06334449969135736 to 0.061794565572465245
Iteration 42 complete. Error: 0.061794565572465245 Total: 0.4048; Orientation: 0.0665; Line Search: 0.2612
th(0)=0.061794565572465245;dx=-1.0494261294592272E-7
New Minimum: 0.061794565572465245 > 0.05945160498289569
END: th(26096.492280877814)=0.05945160498289569; dx=-7.461872919555431E-8 evalInputDelta=0.002342960589569558
Fitness changed from 0.061794565572465245 to 0.05945160498289569
Iteration 43 complete. Error: 0.05945160498289569 Total: 0.4037; Orientation: 0.0659; Line Search: 0.2593
th(0)=0.05945160498289569;dx=-1.227073477016654E-7
New Minimum: 0.05945160498289569 > 0.058440704496777796
WOLF (strong): th(56223.18825807244)=0.058440704496777796; dx=8.674707872944997E-8 evalInputDelta=0.001010900486117891
New Minimum: 0.058440704496777796 > 0.057474130283746344
END: th(28111.59412903622)=0.057474130283746344; dx=-1.798013448610772E-8 evalInputDelta=0.0019774746991493425
Fitness changed from 0.05945160498289569 to 0.057474130283746344
Iteration 44 complete. Error: 0.057474130283746344 Total: 0.5385; Orientation: 0.0656; Line Search: 0.3949
th(0)=0.057474130283746344;dx=-2.2686273819366332E-7
Armijo: th(60564.59358369227)=0.0726116778473339; dx=7.267438293684751E-7 evalInputDelta=-0.015137547563587553
Armijo: th(30282.296791846136)=0.05782355479014753; dx=2.4994054558740585E-7 evalInputDelta=-3.4942450640118583E-4
New Minimum: 0.057474130283746344 > 0.05598630528023728
END: th(10094.098930615379)=0.05598630528023728; dx=-6.792831026664029E-8 evalInputDelta=0.0014878250035090662
Fitness changed from 0.057474130283746344 to 0.05598630528023728
Iteration 45 complete. Error: 0.05598630528023728 Total: 0.6457; Orientation: 0.0667; Line Search: 0.5010
th(0)=0.05598630528023728;dx=-8.367203909381448E-8
New Minimum: 0.05598630528023728 > 0.05422984305345102
WOLFE (weak): th(21747.076900731514)=0.05422984305345102; dx=-7.786343850659572E-8 evalInputDelta=0.0017564622267862567
New Minimum: 0.05422984305345102 > 0.05259970091032064
END: th(43494.15380146303)=0.05259970091032064; dx=-7.205483791937696E-8 evalInputDelta=0.0033866043699166404
Fitness changed from 0.05598630528023728 to 0.05259970091032064
Iteration 46 complete. Error: 0.05259970091032064 Total: 0.5446; Orientation: 0.0674; Line Search: 0.3987
th(0)=0.05259970091032064;dx=-9.712503274825811E-8
Armijo: th(93705.31376345408)=0.05480481025494138; dx=1.4418979900446467E-7 evalInputDelta=-0.0022051093446207454
New Minimum: 0.05259970091032064 > 0.05087569532948551
WOLF (strong): th(46852.65688172704)=0.05087569532948551; dx=2.3532383128103273E-8 evalInputDelta=0.0017240055808351307
END: th(15617.552293909013)=0.0513969078826766; dx=-5.690589412280431E-8 evalInputDelta=0.0012027930276440404
Fitness changed from 0.05259970091032064 to 0.05087569532948551
Iteration 47 complete. Error: 0.05087569532948551 Total: 0.6518; Orientation: 0.0667; Line Search: 0.5071
th(0)=0.05087569532948551;dx=-4.567430667366109E-7
Armijo: th(33646.9964353846)=0.05973520988255912; dx=9.833585445898781E-7 evalInputDelta=-0.008859514553073614
New Minimum: 0.05087569532948551 > 0.04924856588316057
WOLF (strong): th(16823.4982176923)=0.04924856588316057; dx=2.6330773892663355E-7 evalInputDelta=0.0016271294463249403
New Minimum: 0.04924856588316057 > 0.04898734402007458
END: th(5607.8327392307665)=0.04898734402007458; dx=-2.1672613151552945E-7 evalInputDelta=0.0018883513094109239
Fitness changed from 0.05087569532948551 to 0.04898734402007458
Iteration 48 complete. Error: 0.04898734402007458 Total: 0.6605; Orientation: 0.0677; Line Search: 0.5150
th(0)=0.04898734402007458;dx=-1.240947437079604E-7
New Minimum: 0.04898734402007458 > 0.04795526064970697
END: th(12081.709389295287)=0.04795526064970697; dx=-4.6755810151921226E-8 evalInputDelta=0.0010320833703676147
Fitness changed from 0.04898734402007458 to 0.04795526064970697
Iteration 49 complete. Error: 0.04795526064970697 Total: 0.4090; Orientation: 0.0692; Line Search: 0.2607
th(0)=0.04795526064970697;dx=-7.079975672672877E-8
New Minimum: 0.04795526064970697 > 0.04636149429853231
END: th(26029.25382318169)=0.04636149429853231; dx=-5.1659869815100284E-8 evalInputDelta=0.0015937663511746553
Fitness changed from 0.04795526064970697 to 0.04636149429853231
Iteration 50 complete. Error: 0.04636149429853231 Total: 0.4192; Orientation: 0.0672; Line Search: 0.2731
th(0)=0.04636149429853231;dx=-8.23051238084923E-8
New Minimum: 0.04636149429853231 > 0.04546465837065174
WOLF (strong): th(56078.32739230767)=0.04546465837065174; dx=5.032000693413995E-8 evalInputDelta=8.968359278805765E-4
New Minimum: 0.04546465837065174 > 0.04498340189681291
END: th(28039.163696153835)=0.04498340189681291; dx=-1.5992558437176175E-8 evalInputDelta=0.0013780924017194052
Fitness changed from 0.04636149429853231 to 0.04498340189681291
Iteration 51 complete. Error: 0.04498340189681291 Total: 0.5748; Orientation: 0.0737; Line Search: 0.4152
th(0)=0.04498340189681291;dx=-1.4842194852702583E-7
Armijo: th(60408.54694647644)=0.05424059322706055; dx=4.5490809322604515E-7 evalInputDelta=-0.00925719133024764
Armijo: th(30204.27347323822)=0.04505621116800423; dx=1.532430723495096E-7 evalInputDelta=-7.280927119131947E-5
New Minimum: 0.04498340189681291 > 0.04399527467744724
END: th(10068.091157746074)=0.04399527467744724; dx=-4.786694156818068E-8 evalInputDelta=9.881272193656698E-4
Fitness changed from 0.04498340189681291 to 0.04399527467744724
Iteration 52 complete. Error: 0.04399527467744724 Total: 0.7364; Orientation: 0.1304; Line Search: 0.5200
Final threshold in iteration 52: 0.04399527467744724 (> 0.0) after 30.344s (< 30.000s)
Returns
0.04399527467744724
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.04772268638217749, 0.9597752189058228, -0.3405106927964265 ], [ -0.45610075047560045, 0.0521191990818892, -0.026826713281616472 ], [ 1.5327493974521387, 0.7813224634969814, -0.09118845995501634 ], [ 1.5746432408972735, -0.6747745324785115, -0.23114327805480814 ], [ -0.052789198040698086, -0.664207108444809, 0.16327993418611197 ], [ -0.8548293038773342, 0.5106389533944663, -1.0058346239148181 ], [ -1.3502791754816272, -0.12151992353863464, 0.990007177298333 ], [ 1.6749372547993249, -0.3806022767762978, 1.8643297176095175 ], ... ],
[ [ 1.4044498327612833, 0.04059917184526648, -0.7848709421529213 ], [ -1.2408348312887374, 1.5085587011712056, -0.18505583193504388 ], [ -0.06646290502975305, 0.1931639862047153, -1.1376865317413696 ], [ 1.5900731982931082, 1.3141528037182217, 0.07957620432285571 ], [ 0.46335082101716296, 0.020927582883436064, -1.0955538462078338 ], [ -0.6619310844580965, 1.6062359255587981, 2.1662068620261237 ], [ -0.16405923331494496, 1.3603645924819454, -1.4302696464171778 ], [ -1.3966155206282567, -1.3141741551561683, 0.02090564187752221 ], ... ],
[ [ 1.9609668827399718, 0.568263160072272, 1.141991632746237 ], [ 0.751151567233952, 0.9287432919142066, 0.9582782110072927 ], [ 0.2803834890517326, 1.1701399283452019, -1.4928493992514456 ], [ -1.063132281180007, 0.7120298583551363, -1.9540594006618004 ], [ -0.08831659271902763, 0.2527741167777679, -1.5898727989995685 ], [ 0.8634056085428901, 1.691979770655028, 1.4366379821859365 ], [ -0.7731753825365019, -0.5363181287432411, -0.2745024895883763 ], [ 0.25896904014908106, -1.3165846137404615, -1.9871122881923193 ], ... ],
[ [ -0.06139906958235439, -1.2112856877589078, -0.1386697388944232 ], [ 0.7346367149342135, -0.4541412849143288, -1.408124115368609 ], [ 2.2029463499548503, 1.0106149806039635, 0.4727825273484124 ], [ 1.7748497648068091, 0.7766023652569322, 0.4030564391855272 ], [ 1.3220638637383937, 0.8155757734020729, -0.3377164964604932 ], [ 0.8976434766942342, -1.2000261970719766, -0.43697083806669973 ], [ 1.3120606363111702, -1.3680258671790493, -1.0021244715449051 ], [ -1.0559688402307346, -1.0097826529101095, 1.2626104032561407 ], ... ],
[ [ -2.2106890371365226, -0.4975051466031327, -0.758570720792753 ], [ 0.48872890644994993, -0.4868921616180894, 2.2804712358412353 ], [ 2.2518344915698276, 2.155736866259723, -1.5268889360948537 ], [ 0.36694833299375645, -1.5653029716796274, 0.3745313737075267 ], [ 1.1152802375939275, 1.296728378835082, 0.8343780621923181 ], [ -0.22609414021028593, -0.6854139813613747, -0.13960094292804182 ], [ 0.5925634722785487, -0.5893893867701125, 2.227539646956012 ], [ 1.3879879643695154, 2.151946134106587, -0.36381460779398034 ], ... ],
[ [ 0.6168897552841831, 0.18456504513216127, -0.04585073068961827 ], [ 2.1359132053974235, 0.5182314015882639, -0.006232967350070881 ], [ 0.5324529174481132, -0.3782256468135169, 0.8957813518725439 ], [ -0.31901788696086314, 1.766458112754796, 1.364715895687309 ], [ 0.6840278169602344, 1.2124191214489486, -0.6638723015707568 ], [ -1.3081701289002474, -1.0839288762168466, -0.9340744376013221 ], [ -0.5014288463627166, -1.4680214630410333, -0.13722138593469788 ], [ 1.7577603962346553, 1.4925147845806725, -1.3029651555892876 ], ... ],
[ [ 1.7984696502970423, -0.9017469021722189, 0.08239464649705346 ], [ 1.6530492078084582, 0.4423439331266361, 1.245009842113958 ], [ 0.5549534239236649, 0.14388842619084116, -1.7507518867122793 ], [ -1.303843139281202, -0.4358168321168833, 0.6583086473042153 ], [ -1.4238730471449492, 0.19320432445440044, 0.909135317196903 ], [ 0.1409628282306867, 0.9492441968905736, 0.39321623937026695 ], [ -1.3511312566632956, -1.1646339032818633, -0.2647044502228849 ], [ -0.07111754262264657, -2.0783971482527237, -2.3671867548311765 ], ... ],
[ [ -0.42664922173836883, 0.5836828400869283, -1.4993035810010453 ], [ -1.3429384373979367, 0.013829403654693028, 0.02614151029636609 ], [ -1.0741501857353974, 2.32135872842914, -0.11235643319605111 ], [ -1.3422337282359686, -0.7503471081639085, 2.2340838783454036 ], [ 0.04991820353842695, 0.5964613355998944, 0.3049638619899719 ], [ -0.38104082856114413, -1.459653053350555, 0.7274618876416302 ], [ 1.8601975537195918, -2.0225655552348605, 0.8779380478420621 ], [ 0.4818334030219254, 1.2430271569130802, 1.7665276596016855 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.03 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 4.536650071810007, -2.4679709070129348, -3.4887670563486037 ], [ 2.6263419624862148, 4.9507182227696935, 7.482198265194051 ], [ 4.204335920831255, -4.410572500680167, 0.9950470094711953 ], [ -7.372623265522362, 2.9047271944144026, -2.500868323599349 ], [ 5.3754508375502565, -6.998992450554267, -4.664841326352489 ], [ -4.713028249044572, 2.675998922209218, 1.2069109388457266 ], [ 11.263175571761975, -1.4118238966346068, -1.0538033492557402 ], [ 1.3080706721652013, 2.6713953347553314, 9.452406274519767 ], ... ],
[ [ -4.362693025528581, 4.542543937787424, -5.124519068002343 ], [ 3.580047123934078, -6.813700472976305, -6.173830691497401 ], [ 11.59734994656381, 2.4471129991376257, -1.2516414799684512 ], [ 3.762764916286163, -6.358171113602287, -2.852204950660215 ], [ 2.3147426585016335, -1.7289141319457995, 0.08118184033692305 ], [ -3.578205865515187, 2.5602781825849728, -1.925758039952321 ], [ -9.269974498465416, -13.288134028194168, -5.3649459225189755 ], [ 13.22092367074894, 3.0296553985134165, 8.753790317434465 ], ... ],
[ [ -0.5428057884811823, 6.396680193208049, 2.5115133971304884 ], [ -1.1006101206615257, -9.346221613687062, -4.90819907889701 ], [ -8.704611398966664, -2.515128495737066, -9.83902027740694 ], [ -0.00956589660078268, -1.7012087919404055, -6.69737742601556 ], [ -0.2395647299870229, 8.436023523394464, -5.074820620359556 ], [ -1.952980379963897, -5.833393333501305, -0.72603564283392 ], [ -7.668909345839009, 6.033723169963296, -2.270688301399109 ], [ -4.50046427070954, 11.555469487008299, 8.010077699630948 ], ... ],
[ [ 2.362460839086626, -7.008430261615635, 1.2040226358668864 ], [ 10.534632968098546, 9.122378268822251, 4.908479509918435 ], [ 4.64036038095449, 3.812270941318922, -10.645180032424177 ], [ -6.330515800798794, -0.01151601380678595, -0.49015953294459275 ], [ -16.818612763995013, -3.605347694658666, -8.616231990086261 ], [ -6.903873437810729, -3.896189672780918, -0.18052758307865885 ], [ -3.6601709301049583, 10.154476128584141, 2.821378827101622 ], [ -2.3933174176988197, -4.592346786399807, -10.927336929682097 ], ... ],
[ [ 1.4564507067100176, 4.742699341114946, 2.2693341847329864 ], [ 13.839530005832213, 5.953864301516305, 0.9933302934419641 ], [ -7.5884592855937685, -10.771513454417127, -8.816048537900645 ], [ 5.722486581409279, 5.39816550761903, 0.8181580504596222 ], [ 4.761155339042219, -12.33788279026679, -17.55613163034539 ], [ -2.6821030231632155, -0.9387809419386938, -1.4691300100833868 ], [ 3.4712465362474, 7.937463926571391, 8.491968966846555 ], [ -3.30837322738553, -0.5766509897670026, 0.6067411899453847 ], ... ],
[ [ -4.301722640122211, 4.589889940314002, 0.17445794809476553 ], [ -2.3587837278467907, 6.331119545588362, 5.517900139982935 ], [ 1.0022479990008368, 2.334977939470801, -14.50149815123793 ], [ 3.6585999669511815, -8.844415400831002, -0.9254284759245932 ], [ -3.7666034721363517, -8.275852229477389, 0.7040023246840549 ], [ -2.280989409867863, -3.9955472505466703, -7.108702894372053 ], [ 18.35382148456102, 0.19316132423562615, 7.948394746141694 ], [ 7.74142267683857, -6.939806986197034, -13.379081419444372 ], ... ],
[ [ 6.498483985946696, 0.40881617370143236, -1.0444040194066502 ], [ 2.3221277415626944, -7.959771334542651, -3.4363418044014713 ], [ -3.429234601468956, -13.818693312238944, -10.105574091965 ], [ 6.967378376768884, -3.8584944332233264, 0.4510806754133776 ], [ 3.6373090802046097, 6.517798059552063, -3.7516367560681347 ], [ -12.152456610429722, -3.2822202555980997, -1.2987343029152796 ], [ 4.451883008424948, -4.340707569819506, 5.536669841521972 ], [ -0.7683078375443276, 7.560140658819295, -1.0631115138957374 ], ... ],
[ [ 1.4586816949206043, -10.259908841254553, -7.153577476754415 ], [ 12.088840717751571, 1.7293106181647744, 4.419674000025213 ], [ 0.2319252489368479, -4.679041887995523, -12.388395333852072 ], [ 2.617534274189912, 0.06503192773759203, 12.349976981617887 ], [ -3.406852048278961, 0.8861995454601774, 9.371312939359834 ], [ -3.4154188707677986, 6.423056534216818, 10.742218829355403 ], [ 0.4628121720315484, 14.261228269849843, 2.952985466414787 ], [ 3.404562931930794, 7.789173129897753, 0.21867818830658822 ], ... ],
...
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 30.23 seconds (0.326 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13500285221547
Reset training subject: 13500365249542
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=94.4107808179455}, derivative=-0.012035856170180149}
New Minimum: 94.4107808179455 > 94.41078081794429
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=94.41078081794429}, derivative=-0.012035856170180059}, evalInputDelta = -1.2079226507921703E-12
New Minimum: 94.41078081794429 > 94.41078081793705
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=94.41078081793705}, derivative=-0.012035856170179509}, evalInputDelta = -8.44124770082999E-12
New Minimum: 94.41078081793705 > 94.41078081788652
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=94.41078081788652}, derivative=-0.012035856170175666}, evalInputDelta = -5.897504706808832E-11
New Minimum: 94.41078081788652 > 94.41078081753265
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=94.41078081753265}, derivative=-0.012035856170148754}, evalInputDelta = -4.128395403313334E-10
New Minimum: 94.41078081753265 > 94.41078081505569
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=94.41078081505569}, derivative=-0.012035856169960373}, evalInputDelta = -2.889805728045758E-9
New Minimum: 94.41078081505569 > 94.41078079771682
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=94.41078079771682}, derivative=-0.012035856168641714}, evalInputDelta = -2.0228668518029735E-8
New Minimum: 94.41078079771682 > 94.41078067634484
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=94.41078067634484}, derivative=-0.0120358561594111}, evalInputDelta = -1.4160065120449872E-7
New Minimum: 94.41078067634484 > 94.41077982674099
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=94.41077982674099}, derivative=-0.012035856094796796}, evalInputDelta = -9.912045015880722E-7
New Minimum: 94.41077982674099 > 94.41077387951407
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=94.41077387951407}, derivative=-0.012035855642496665}, evalInputDelta = -6.938431425851377E-6
New Minimum: 94.41077387951407 > 94.41073224893196
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=94.41073224893196}, derivative=-0.012035852476395745}, evalInputDelta = -4.85690135292316E-5
New Minimum: 94.41073224893196 > 94.41044083516383
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=94.41044083516383}, derivative=-0.012035830313689325}, evalInputDelta = -3.399827816679135E-4
New Minimum: 94.41044083516383 > 94.40840095381185
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=94.40840095381185}, derivative=-0.012035675174744378}, evalInputDelta = -0.002379864133644105
New Minimum: 94.40840095381185 > 94.39412252057278
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=94.39412252057278}, derivative=-0.01203458920212974}, evalInputDelta = -0.016658297372714514
New Minimum: 94.39412252057278 > 94.29420956292073
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=94.29420956292073}, derivative=-0.012026987393827272}, evalInputDelta = -0.11657125502476617
New Minimum: 94.29420956292073 > 93.59658653539734
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=93.59658653539734}, derivative=-0.011973774735710006}, evalInputDelta = -0.8141942825481578
New Minimum: 93.59658653539734 > 88.79984146874014
F(474.7561509943) = LineSearchPoint{point=PointSample{avg=88.79984146874014}, derivative=-0.011601286128889145}, evalInputDelta = -5.610939349205353
New Minimum: 88.79984146874014 > 59.46681617646382
F(3323.2930569601003) = LineSearchPoint{point=PointSample{avg=59.46681617646382}, derivative=-0.008993865881143117}, evalInputDelta = -34.94396464148167
F(23263.0513987207) = LineSearchPoint{point=PointSample{avg=62.100957672407006}, derivative=0.009258075853079075}, evalInputDelta = -32.30982314553849
62.100957672407006 <= 94.4107808179455
New Minimum: 59.46681617646382 > 15.281940720656253
F(13148.851062766538) = LineSearchPoint{point=PointSample{avg=15.281940720656253}, derivative=-8.809142651444724E-19}, evalInputDelta = -79.12884009728924
Left bracket at 13148.851062766538
Converged to left
Fitness changed from 94.4107808179455 to 15.281940720656253
Iteration 1 complete. Error: 15.281940720656253 Total: 3.3402; Orientation: 0.0682; Line Search: 3.0358
F(0.0) = LineSearchPoint{point=PointSample{avg=15.281940720656253}, derivative=-0.0010466116997471027}
New Minimum: 15.281940720656253 > 6.858971650176087
F(13148.851062766538) = LineSearchPoint{point=PointSample{avg=6.858971650176087}, derivative=-2.345601730306487E-4}, evalInputDelta = -8.422969070480168
F(92041.95743936577) = LineSearchPoint{point=PointSample{avg=180.5495934091187}, derivative=0.004637748987268075}, evalInputDelta = 165.26765268846245
F(7080.150572258905) = LineSearchPoint{point=PointSample{avg=9.419700356185727}, derivative=-6.093531853613198E-4}, evalInputDelta = -5.8622403644705265
F(49561.05400581234) = LineSearchPoint{point=PointSample{avg=39.259236707287755}, derivative=0.002014197900953378}, evalInputDelta = 23.977295986631503
F(3812.388769677872) = LineSearchPoint{point=PointSample{avg=11.740657574474342}, derivative=-8.111648073855274E-4}, evalInputDelta = -3.541283146181911
F(26686.721387745103) = LineSearchPoint{point=PointSample{avg=9.342875551574199}, derivative=6.015165467839254E-4}, evalInputDelta = -5.939065169082054
9.342875551574199 <= 15.281940720656253
New Minimum: 6.858971650176087 > 6.413538259141767
F(16946.881949929277) = LineSearchPoint{point=PointSample{avg=6.413538259141767}, derivative=1.5246593050577406E-19}, evalInputDelta = -8.868402461514487
Right bracket at 16946.881949929277
Converged to right
Fitness changed from 15.281940720656253 to 6.413538259141767
Iteration 2 complete. Error: 6.413538259141767 Total: 1.3136; Orientation: 0.0672; Line Search: 1.1686
F(0.0) = LineSearchPoint{point=PointSample{avg=6.413538259141767}, derivative=-3.586769132561857E-4}
New Minimum: 6.413538259141767 > 3.7779063560616195
F(16946.881949929277) = LineSearchPoint{point=PointSample{avg=3.7779063560616195}, derivative=4.763067939826402E-5}, evalInputDelta = -2.635631903080147
3.7779063560616195 <= 6.413538259141767
New Minimum: 3.7779063560616195 > 3.730593580851191
F(14960.23066516287) = LineSearchPoint{point=PointSample{avg=3.730593580851191}, derivative=2.456395547037471E-20}, evalInputDelta = -2.6829446782905757
Right bracket at 14960.23066516287
Converged to right
Fitness changed from 6.413538259141767 to 3.730593580851191
Iteration 3 complete. Error: 3.730593580851191 Total: 0.8017; Orientation: 0.0663; Line Search: 0.6575
F(0.0) = LineSearchPoint{point=PointSample{avg=3.730593580851191}, derivative=-1.3336396207920399E-4}
New Minimum: 3.730593580851191 > 2.5608681541677516
F(14960.23066516287) = LineSearchPoint{point=PointSample{avg=2.5608681541677516}, derivative=-2.301403139750347E-5}, evalInputDelta = -1.1697254266834394
F(104721.61465614008) = LineSearchPoint{point=PointSample{avg=30.210584348609544}, derivative=6.390855526926996E-4}, evalInputDelta = 26.479990767758352
F(8055.508819703084) = LineSearchPoint{point=PointSample{avg=2.895604926515478}, derivative=-7.39447686352114E-5}, evalInputDelta = -0.8349886543357128
F(56388.561737921584) = LineSearchPoint{point=PointSample{avg=7.937361574303448}, derivative=2.8257039202874405E-4}, evalInputDelta = 4.206767993452257
F(4337.581672147814) = LineSearchPoint{point=PointSample{avg=3.2215068582573916}, derivative=-1.0136901176320797E-4}, evalInputDelta = -0.5090867225937994
F(30363.071705034698) = LineSearchPoint{point=PointSample{avg=3.081381434625424}, derivative=9.060069013276804E-5}, evalInputDelta = -0.649212146225767
3.081381434625424 <= 3.730593580851191
New Minimum: 2.5608681541677516 > 2.5249659070449786
F(18080.261789016105) = LineSearchPoint{point=PointSample{avg=2.5249659070449786}, derivative=1.3976043629695956E-20}, evalInputDelta = -1.2056276738062124
Right bracket at 18080.261789016105
Converged to right
Fitness changed from 3.730593580851191 to 2.5249659070449786
Iteration 4 complete. Error: 2.5249659070449786 Total: 1.2844; Orientation: 0.0707; Line Search: 1.1334
F(0.0) = L
...skipping 18585 bytes...
54039550687 Total: 1.3741; Orientation: 0.0665; Line Search: 1.2300
F(0.0) = LineSearchPoint{point=PointSample{avg=0.3100854039550687}, derivative=-2.511137601509103E-6}
New Minimum: 0.3100854039550687 > 0.2902638614406976
F(19593.07569267249) = LineSearchPoint{point=PointSample{avg=0.2902638614406976}, derivative=4.878164215900128E-7}, evalInputDelta = -0.01982154251437107
0.2902638614406976 <= 0.3100854039550687
New Minimum: 0.2902638614406976 > 0.2894865131550546
F(16406.023140774843) = LineSearchPoint{point=PointSample{avg=0.2894865131550546}, derivative=2.448454613156962E-22}, evalInputDelta = -0.02059889080001409
Right bracket at 16406.023140774843
Converged to right
Fitness changed from 0.3100854039550687 to 0.2894865131550546
Iteration 21 complete. Error: 0.2894865131550546 Total: 0.6647; Orientation: 0.0667; Line Search: 0.5169
F(0.0) = LineSearchPoint{point=PointSample{avg=0.2894865131550546}, derivative=-1.8899282644167232E-6}
New Minimum: 0.2894865131550546 > 0.2714493427019575
F(16406.023140774843) = LineSearchPoint{point=PointSample{avg=0.2714493427019575}, derivative=-3.0891910990743676E-7}, evalInputDelta = -0.018037170453097107
F(114842.16198542391) = LineSearchPoint{point=PointSample{avg=0.7079258482511556}, derivative=9.17713581714828E-6}, evalInputDelta = 0.418439335096101
F(8834.012460417223) = LineSearchPoint{point=PointSample{avg=0.2765511164716249}, derivative=-1.0386156427578767E-6}, evalInputDelta = -0.01293539668342969
F(61838.08722292056) = LineSearchPoint{point=PointSample{avg=0.3568693688273855}, derivative=4.069260087195202E-6}, evalInputDelta = 0.06738285567233093
F(4756.775940224658) = LineSearchPoint{point=PointSample{avg=0.2815867987725194}, derivative=-1.431529160446575E-6}, evalInputDelta = -0.007899714382535172
F(33297.43158157261) = LineSearchPoint{point=PointSample{avg=0.27997905088180913}, derivative=1.3188654633743136E-6}, evalInputDelta = -0.009507462273245448
0.27997905088180913 <= 0.2894865131550546
New Minimum: 0.2714493427019575 > 0.27095420215832955
F(19611.655474600273) = LineSearchPoint{point=PointSample{avg=0.27095420215832955}, derivative=-9.26442286059391E-23}, evalInputDelta = -0.018532310996725032
Left bracket at 19611.655474600273
F(26454.54352808644) = LineSearchPoint{point=PointSample{avg=0.2732104143391994}, derivative=6.594327316871565E-7}, evalInputDelta = -0.016276098815855178
Right bracket at 26454.54352808644
Converged to left
Fitness changed from 0.2894865131550546 to 0.27095420215832955
Iteration 22 complete. Error: 0.27095420215832955 Total: 1.5259; Orientation: 0.0666; Line Search: 1.3804
F(0.0) = LineSearchPoint{point=PointSample{avg=0.27095420215832955}, derivative=-2.039246793318434E-6}
New Minimum: 0.27095420215832955 > 0.2548477734512882
F(19611.655474600273) = LineSearchPoint{point=PointSample{avg=0.2548477734512882}, derivative=3.9671042224039436E-7}, evalInputDelta = -0.016106428707041376
0.2548477734512882 <= 0.27095420215832955
New Minimum: 0.2548477734512882 > 0.2542142512720571
F(16417.77831022775) = LineSearchPoint{point=PointSample{avg=0.2542142512720571}, derivative=-9.26442286059391E-23}, evalInputDelta = -0.016739950886272448
Left bracket at 16417.77831022775
Converged to left
Fitness changed from 0.27095420215832955 to 0.2542142512720571
Iteration 23 complete. Error: 0.2542142512720571 Total: 0.6602; Orientation: 0.0658; Line Search: 0.5168
F(0.0) = LineSearchPoint{point=PointSample{avg=0.2542142512720571}, derivative=-1.5468953287629865E-6}
New Minimum: 0.2542142512720571 > 0.23943965621466443
F(16417.77831022775) = LineSearchPoint{point=PointSample{avg=0.23943965621466443}, derivative=-2.5293346392921096E-7}, evalInputDelta = -0.014774595057392675
F(114924.44817159425) = LineSearchPoint{point=PointSample{avg=0.5969156456836344}, derivative=7.510837725073441E-6}, evalInputDelta = 0.3427013944115773
F(8840.342167045712) = LineSearchPoint{point=PointSample{avg=0.24361891570971353}, derivative=-8.501466323140305E-7}, evalInputDelta = -0.010595335562343572
F(61882.395169319985) = LineSearchPoint{point=PointSample{avg=0.30939633683774087}, derivative=3.330345546379706E-6}, evalInputDelta = 0.05518208556568377
F(4760.184243793845) = LineSearchPoint{point=PointSample{avg=0.24774368931539825}, derivative=-1.1717229537520101E-6}, evalInputDelta = -0.006470561956658849
F(33321.28970655691) = LineSearchPoint{point=PointSample{avg=0.24642399976835816}, derivative=1.079311296313848E-6}, evalInputDelta = -0.007790251503698942
0.24642399976835816 <= 0.2542142512720571
New Minimum: 0.23943965621466443 > 0.2390337969186488
F(19626.996178917583) = LineSearchPoint{point=PointSample{avg=0.2390337969186488}, derivative=-1.3896634290890865E-22}, evalInputDelta = -0.015180454353408313
Left bracket at 19626.996178917583
Converged to left
Fitness changed from 0.2542142512720571 to 0.2390337969186488
Iteration 24 complete. Error: 0.2390337969186488 Total: 1.3384; Orientation: 0.0659; Line Search: 1.1958
F(0.0) = LineSearchPoint{point=PointSample{avg=0.2390337969186488}, derivative=-1.6815864612899976E-6}
New Minimum: 0.2390337969186488 > 0.22574536781043902
F(19626.996178917583) = LineSearchPoint{point=PointSample{avg=0.22574536781043902}, derivative=3.2748938121988425E-7}, evalInputDelta = -0.013288429108209765
0.22574536781043902 <= 0.2390337969186488
New Minimum: 0.22574536781043902 > 0.2252214997077713
F(16427.697925543092) = LineSearchPoint{point=PointSample{avg=0.2252214997077713}, derivative=2.2499312661442353E-22}, evalInputDelta = -0.013812297210877483
Right bracket at 16427.697925543092
Converged to right
Fitness changed from 0.2390337969186488 to 0.2252214997077713
Iteration 25 complete. Error: 0.2252214997077713 Total: 0.6633; Orientation: 0.0674; Line Search: 0.5183
F(0.0) = LineSearchPoint{point=PointSample{avg=0.2252214997077713}, derivative=-1.2840181563324334E-6}
New Minimum: 0.2252214997077713 > 0.21294966634094104
F(16427.697925543092) = LineSearchPoint{point=PointSample{avg=0.21294966634094104}, derivative=-2.100236044122723E-7}, evalInputDelta = -0.01227183336683027
F(114993.88547880165) = LineSearchPoint{point=PointSample{avg=0.5098270856650511}, derivative=6.2339437071086955E-6}, evalInputDelta = 0.2846055859572798
F(8845.683498369357) = LineSearchPoint{point=PointSample{avg=0.21642123192134144}, derivative=-7.05713397606193E-7}, evalInputDelta = -0.008800267786429866
F(61919.7844885855) = LineSearchPoint{point=PointSample{avg=0.271045143289919}, derivative=2.764115154751251E-6}, evalInputDelta = 0.045823643582147716
F(4763.060345275808) = LineSearchPoint{point=PointSample{avg=0.2198472400234383}, derivative=-9.72623286249073E-7}, evalInputDelta = -0.005374259684332999
F(33341.422416930654) = LineSearchPoint{point=PointSample{avg=0.21874872562318404}, derivative=8.957459342510889E-7}, evalInputDelta = -0.006472774084587268
0.21874872562318404 <= 0.2252214997077713
New Minimum: 0.21294966634094104 > 0.21261231623606439
F(19640.19497624978) = LineSearchPoint{point=PointSample{avg=0.21261231623606439}, derivative=-8.602678370551488E-23}, evalInputDelta = -0.01260918347170692
Left bracket at 19640.19497624978
Converged to left
Fitness changed from 0.2252214997077713 to 0.21261231623606439
Iteration 26 complete. Error: 0.21261231623606439 Total: 1.4725; Orientation: 0.0660; Line Search: 1.3294
F(0.0) = LineSearchPoint{point=PointSample{avg=0.21261231623606439}, derivative=-1.4046510124043147E-6}
New Minimum: 0.21261231623606439 > 0.20150723443906665
F(19640.19497624978) = LineSearchPoint{point=PointSample{avg=0.20150723443906665}, derivative=2.7379851216928327E-7}, evalInputDelta = -0.011105081796997734
0.20150723443906665 <= 0.21261231623606439
New Minimum: 0.20150723443906665 > 0.20106863337445702
F(16436.37139711776) = LineSearchPoint{point=PointSample{avg=0.20106863337445702}, derivative=3.970466940254533E-23}, evalInputDelta = -0.011543682861607368
Right bracket at 16436.37139711776
Converged to right
Fitness changed from 0.21261231623606439 to 0.20106863337445702
Iteration 27 complete. Error: 0.20106863337445702 Total: 0.6607; Orientation: 0.0700; Line Search: 0.5139
Final threshold in iteration 27: 0.20106863337445702 (> 0.0) after 30.224s (< 30.000s)
Returns
0.20106863337445702
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.026476945688550965, 0.7334377339874862, -0.6330259113995018 ], [ -0.22663614603642307, -0.0787556378934933, 0.12564829123124216 ], [ 1.3569984055899356, 0.6577560269239741, -0.27490157568882967 ], [ 1.749753752704061, -0.4472662342592527, -0.049084891028390525 ], [ 0.07727334956748674, -0.8463904469184509, 0.020215003434064908 ], [ -0.648164210079079, 0.2803729537043136, -1.2946032531773115 ], [ -1.3438810044487173, -0.21539354967009253, 1.2355236307777275 ], [ 1.8210738206947654, -0.7413118022420491, 1.7274444854020463 ], ... ],
[ [ 1.2426113395286738, -0.002198316231101986, -1.1313234604465574 ], [ -1.2038728824900011, 1.553484203590473, -0.08093873768805303 ], [ -0.4308100910174723, 0.24798229513141665, -0.9685596858962948 ], [ 1.6592110909361546, 1.3367072574279164, 0.4299482773872014 ], [ 0.5480608299009295, -0.06685264674157171, -1.0037857235077112 ], [ -0.8900580135129192, 1.4886754470791932, 1.9889616472195428 ], [ -0.19573106629191134, 1.4947334443130602, -1.1556305628083219 ], [ -1.3781747443715648, -1.1703045012502815, -0.02556112780740507 ], ... ],
[ [ 1.7820638675091942, 0.27443015943826815, 1.2308159198827389 ], [ 0.669995905197216, 0.9386975266077121, 0.8610056476443573 ], [ 0.3137446554386125, 1.3373846721410858, -1.3566116366419174 ], [ -0.7254323497741766, 0.48980105259140044, -2.090483572278535 ], [ 0.033123256325656984, -0.022623982644724188, -1.7503671175234414 ], [ 0.5727406928149696, 1.4452640383291864, 1.0346259570738332 ], [ -0.913655356292443, -0.07109485944919415, -0.2883913439796159 ], [ 0.2730743841907891, -1.4787707048257852, -1.870336546264784 ], ... ],
[ [ 5.555674586255653E-4, -0.8029093833911579, -0.020440346759846593 ], [ 0.6107409932375223, -0.23937160457207776, -1.3203199035826712 ], [ 2.150412815150715, 1.1079635038210307, 0.676900150491354 ], [ 1.8757617991164957, 0.526092721082758, -0.06996401849155372 ], [ 1.5004490288871934, 0.5638764442782336, -0.6832353074834987 ], [ 0.4578008119971786, -1.1180829527662306, -0.6250918562681731 ], [ 1.2253548925867748, -1.30506758787151, -0.6100424927521294 ], [ -1.1148965559799109, -0.9612293921768172, 1.1884976271087708 ], ... ],
[ [ -2.2413447515652107, -0.5506404754049417, -0.7827793074194429 ], [ 0.6759428394543218, -0.6748107567970564, 2.1161432996347824 ], [ 2.308405181541894, 2.170182329544798, -1.6320387873542495 ], [ 0.39080900221883397, -1.6567851468717898, 0.06283396701364076 ], [ 1.1338394963216378, 1.0433493229593438, 0.7715743298033506 ], [ -0.408455654940171, -0.6452972829091188, 0.11078164844730962 ], [ 0.7362175201536236, -0.38724809692010614, 2.3640957752525056 ], [ 1.457871761643987, 2.1933139974485747, -0.7772289554816283 ], ... ],
[ [ 0.5769736116215661, 0.08064067089670315, 0.03339883646849129 ], [ 2.110791885605608, 0.40110097653175747, -0.16838535497001644 ], [ 0.5491188454763509, -0.2172444556090264, 1.0469187242329607 ], [ -0.5596606420922764, 1.767555713463714, 1.5626272535557884 ], [ 0.875303819938654, 1.2658218671296686, -0.7780394136435704 ], [ -1.2771058672470494, -0.8106740007480225, -0.8323202114630208 ], [ -0.4516914071903643, -0.9357791645354162, 0.19758878769506258 ], [ 1.5653597257812935, 1.3096694837817326, -0.7735415749409964 ], ... ],
[ [ 1.690168310129969, -1.0745908345580832, 0.1533618642148126 ], [ 1.5242562436280067, 0.2591927120383332, 1.1164598685478173 ], [ 0.3797209311070832, -0.16866072693884143, -1.5548763956488039 ], [ -1.4301069152906343, -0.31494685518131876, 0.5231417130098353 ], [ -1.50235728957439, 0.10725040455923061, 0.7850547978901143 ], [ -0.12495377289167137, 0.8026522439864586, 0.4623313730668931 ], [ -1.2399321762484496, -1.3597381543008298, -0.5212054508561275 ], [ 0.19840786399382204, -2.010794474478871, -2.740617930647282 ], ... ],
[ [ -0.34977754075579653, 0.5171176951180816, -1.608887161013927 ], [ -1.3038018269602707, -0.3995113596651552, -0.33490938387223895 ], [ -1.0155794789704915, 2.3175364303689747, -0.8367113553586064 ], [ -1.327832090965522, -0.9251360036675339, 2.157578442680457 ], [ -0.24678383125689748, 0.17197668426285645, -0.25499205282599147 ], [ -0.2169424534016331, -1.3960111379822773, 0.6995613537736453 ], [ 1.6734120298817063, -2.1081307430161877, 0.5851179549311666 ], [ 0.32554586555615006, 1.0750252517290915, 1.5186100604642456 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.03 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 4.898911633263008, -2.4963921506247186, -3.083019859070962 ], [ 2.441839566508121, 4.964713220064459, 7.371161701669591 ], [ 4.159612417046236, -4.35180746940097, 1.0975161045686932 ], [ -7.4094338348559665, 2.6487500106222517, -2.7043582956912693 ], [ 5.60442434720429, -7.264945224745264, -4.5123713306870075 ], [ -5.0679875746998135, 2.6726787645321193, 1.2155943949429824 ], [ 10.754234301300626, -1.3878800235590432, -0.6445541468483137 ], [ 1.6223634850035413, 2.8247552417781576, 8.999146200364493 ], ... ],
[ [ -4.697710139352942, 4.7558719960514155, -5.504326535128011 ], [ 3.468080714427079, -6.918696816215645, -5.623469093452168 ], [ 11.440058938783695, 2.4511406987122175, -1.613416885456404 ], [ 3.7418110107898097, -6.00032726777858, -2.936811095482834 ], [ 2.2294942730000007, -1.5034867849532265, -0.4113158776521581 ], [ -3.2678388154295734, 2.291870457356162, -1.6450430861696248 ], [ -9.51233264334237, -13.714120096686454, -4.979892082222632 ], [ 13.152330014132843, 3.2766354498558257, 8.432407958713606 ], ... ],
[ [ -0.05759648642758877, 6.568429106033489, 2.5291076433228263 ], [ -1.0512967936980169, -9.607429975723791, -4.742830416271637 ], [ -8.797606686165063, -2.8797782859215335, -9.389534768650767 ], [ -0.21427078711392478, -1.8818959905432977, -6.551457263885541 ], [ 0.14188799954139614, 8.672574542737502, -5.173539097401412 ], [ -1.814969964296489, -5.786355280333125, -0.6943250841379821 ], [ -7.877522149763412, 5.6783769360933105, -2.0563536354535645 ], [ -4.796487443546821, 11.250170191627856, 8.138102802369856 ], ... ],
[ [ 2.153307682391576, -7.217175454885703, 1.3588890154963569 ], [ 10.24077401053562, 9.28507774288424, 4.9970533808708515 ], [ 4.298801970985775, 3.130012865439612, -10.23949326072929 ], [ -6.322860752603656, 0.05609098017741547, -0.9973790395986 ], [ -16.828762575652778, -3.450453862562824, -8.910074660322746 ], [ -7.164791660330028, -3.8737934330046873, 0.13454713341375082 ], [ -4.106132547399462, 10.199060410894411, 2.9979174681691503 ], [ -2.092441017969364, -4.961076459058993, -10.935275423820164 ], ... ],
[ [ 1.1967007243899843, 4.607999463113472, 2.3916281742068928 ], [ 14.03209981856091, 6.089915528799568, 0.9172803410672151 ], [ -7.520169839726908, -10.620833472539825, -9.055904215235453 ], [ 5.6568042293829945, 5.680036299440289, 0.634193823474823 ], [ 5.119718422288807, -12.204530896397012, -17.67526549406928 ], [ -2.6311646863144693, -1.484680827952583, -1.0305600588174557 ], [ 3.223020049401697, 7.403842191372541, 9.032494049129983 ], [ -3.2210683951425327, -0.35371747573020895, -0.01021540898867226 ], ... ],
[ [ -4.2060422938823665, 4.333852414574639, 0.3012747766834469 ], [ -2.2091928822848135, 6.636591373426493, 4.974665951015528 ], [ 0.7690691947915329, 2.1999543472504586, -14.271965480781526 ], [ 3.897334549385899, -9.066698845206314, -0.9615974758015309 ], [ -3.982852633382526, -8.265616338572585, 0.9709492952821756 ], [ -2.575719603441923, -3.915410657383867, -6.846906540949647 ], [ 18.067764180456216, 0.0811832424571231, 7.589458566705051 ], [ 7.769121486995302, -7.131315447144322, -13.480654695145311 ], ... ],
[ [ 6.240091318581019, 0.2905180808810308, -1.1015460879393026 ], [ 2.3516358060901443, -8.033421046500388, -3.3361019134688132 ], [ -3.637949436836761, -13.795697054280847, -9.901189932578738 ], [ 6.837893935435138, -3.8121934205695323, 0.3229476076564536 ], [ 3.7136889103382478, 5.981770489860621, -3.5084600221313975 ], [ -12.20988540556538, -3.618943646938089, -1.397242026816964 ], [ 4.385559956440121, -4.092448420471487, 5.5103444035714775 ], [ -0.7051817140511033, 7.365994765459266, -0.7524597246005196 ], ... ],
[ [ 1.3644441642022738, -10.410906472301473, -6.693178767933737 ], [ 12.309576972186179, 1.4714075663159447, 4.390230932970239 ], [ 0.24162000929840446, -4.79508807882285, -12.127839708156364 ], [ 2.371360599120497, -0.14561363345555206, 12.739118624855582 ], [ -3.3306052213415267, 1.5094041745740585, 8.958253017840066 ], [ -3.610831147626966, 6.386966731225892, 10.635743524943608 ], [ 0.853346004104515, 14.308892373940065, 2.6187403259890485 ], [ 3.3968780899197673, 7.643936866761299, 0.1042036816743675 ], ... ],
...
]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 30.39 seconds (0.476 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13530571730098
Reset training subject: 13530649545569
Adding measurement 539c8410 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 94.4107808179455 < 94.4107808179455. Total: 1
th(0)=94.4107808179455;dx=-0.012035856170180149
Adding measurement 1c790628 to history. Total: 1
New Minimum: 94.4107808179455 > 94.38485247623717
WOLFE (weak): th(2.154434690031884)=94.38485247623717; dx=-0.01203388409931757 evalInputDelta=0.025928341708322478
Adding measurement 28773a06 to history. Total: 2
New Minimum: 94.38485247623717 > 94.35892838322671
WOLFE (weak): th(4.308869380063768)=94.35892838322671; dx=-0.012031912028454994 evalInputDelta=0.05185243471878209
Adding measurement 15a5c714 to history. Total: 3
New Minimum: 94.35892838322671 > 94.25527449816366
WOLFE (weak): th(12.926608140191302)=94.25527449816366; dx=-0.012024023745004678 evalInputDelta=0.1555063197818356
Adding measurement 6f4f241a to history. Total: 4
New Minimum: 94.25527449816366 > 93.78967325755974
WOLFE (weak): th(51.70643256076521)=93.78967325755974; dx=-0.01198852646947826 evalInputDelta=0.6211075603857523
Adding measurement 6773595 to history. Total: 5
New Minimum: 93.78967325755974 > 91.32971551579132
WOLFE (weak): th(258.53216280382605)=91.32971551579132; dx=-0.011799207666670693 evalInputDelta=3.0810653021541725
Adding measurement 6e747ed7 to history. Total: 6
New Minimum: 91.32971551579132 > 76.84210774656941
END: th(1551.1929768229563)=76.84210774656941; dx=-0.010615965149123403 evalInputDelta=17.568673071376082
Fitness changed from 94.4107808179455 to 76.84210774656941
Iteration 1 complete. Error: 76.84210774656941 Total: 1.6213; Orientation: 0.0988; Line Search: 1.2902
Non-optimal measurement 76.84210774656941 < 76.84210774656941. Total: 7
Rejected: LBFGS Orientation magnitude: 1.442e+03, gradient 9.684e-02, dot -0.972; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 76.84210774656941, 91.32971551579132, 93.78967325755974, 94.25527449816366, 94.35892838322671, 94.38485247623717, 94.4107808179455
Rejected: LBFGS Orientation magnitude: 1.442e+03, gradient 9.684e-02, dot -0.972; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 76.84210774656941, 91.32971551579132, 93.78967325755974, 94.25527449816366, 94.35892838322671, 94.38485247623717
Rejected: LBFGS Orientation magnitude: 1.442e+03, gradient 9.684e-02, dot -0.972; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 76.84210774656941, 91.32971551579132, 93.78967325755974, 94.25527449816366, 94.35892838322671
Rejected: LBFGS Orientation magnitude: 1.442e+03, gradient 9.684e-02, dot -0.972; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 76.84210774656941, 91.32971551579132, 93.78967325755974, 94.25527449816366
LBFGS Accumulation History: 3 points
Removed measurement 6e747ed7 to history. Total: 6
Removed measurement 6773595 to history. Total: 5
Removed measurement 6f4f241a to history. Total: 4
Removed measurement 15a5c714 to history. Total: 3
Adding measurement 70df9478 to history. Total: 3
th(0)=76.84210774656941;dx=-0.009378147201831128
Adding measurement 6d5f4f2b to history. Total: 4
New Minimum: 76.84210774656941 > 49.38985895885882
END: th(3341.943960201201)=49.38985895885882; dx=-0.007050763105843709 evalInputDelta=27.452248787710595
Fitness changed from 76.84210774656941 to 49.38985895885882
Iteration 2 complete. Error: 49.38985895885882 Total: 12.6648; Orientation: 12.2351; Line Search: 0.3519
Non-optimal measurement 49.38985895885882 < 49.38985895885882. Total: 5
Rejected: LBFGS Orientation magnitude: 1.138e+03, gradient 7.319e-02, dot -0.978; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 49.38985895885882, 76.84210774656941, 94.35892838322671, 94.38485247623717, 94.4107808179455
Rejected: LBFGS Orientation magnitude: 1.138e+03, gradient 7.319e-02, dot -0.978; [b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00, 9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 49.38985895885882, 76.84210774656941, 94.35892838322671, 94.38485247623717
LBFGS Accumulation History: 3 points
Removed measurement 6d5f4f2b to history. Total: 4
Removed measurement 70df9478 to history. Total: 3
Adding measurement f19b397 to history. Total: 3
th(0)=49.38985895885882;dx=-0.005356624771513699
Adding measurement 349c6289 to history. Total: 4
New Minimum: 49.38985895885882 > 20.460720238237112
END: th(7200.000000000001)=20.460720238237112; dx=-0.00267924709532566 evalInputDelta=28.929138720621705
Fitness changed from 49.38985895885882 to 20.460720238237112
Iteration 3 complete. Error: 20.460720238237112 Total: 5.4355; Orientation: 4.9693; Line Search: 0.3871
Non-optimal measurement 20.460720238237112 < 20.460720238237112. Total: 5
Rejected: LBFGS Orientation magnitude: 6.744e+02, gradient 3.879e-02, dot -0.994; [9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00, b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00]
Orientation rejected. Popping history element from 20.460720238237112, 49.38985895885882, 94.35892838322671, 94.38485247623717, 94.4107808179455
Rejected: LBFGS Orientation magnitude: 6.744e+02, gradient 3.879e-02, dot -0.994; [b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00, 9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 20.460720238237112, 49.38985895885882, 94.35892838322671, 94.38485247623717
LBFGS Accumulation History: 3 points
Removed measurement 349c6289 to history. Total: 4
Removed measurement f19b397 to history. Total: 3
Adding measurement 92829d3 to history. Total: 3
th(0)=20.460720238237112;dx=-0.0015043792851828848
Adding measurement 2a611ca to history. Total: 4
New Minimum: 20.460720238237112 > 6.708659089542602
END: th(15511.929768229566)=6.708659089542602; dx=-2.687155333432413E-4 evalInputDelta=13.75206114869451
Fitness changed from 20.460720238237112 to 6.708659089542602
Iteration 4 complete. Error: 6.708659089542602 Total: 5.2227; Orientation: 4.7613; Line Search: 0.3790
Non-optimal measurement 6.708659089542602 < 6.708659089542602. Total: 5
Rejected: LBFGS Orientation magnitude: 2.994e+02, gradient 1.617e-02, dot -0.962; [b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00, 9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.708659089542602, 20.460720238237112, 94.35892838322671, 94.38485247623717, 94.4107808179455
Rejected: LBFGS Orientation magnitude: 2.994e+02, gradient 1.617e-02, dot -0.962; [b286f763-8b7d-4844-8487-ce06aa103f2a = 1.000/1.000e+00, 9cf775f3-e7fc-49fe-9b06-c0024a440416 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.708659089542602, 20.460720238237112, 94.35892838322671, 94.38485247623717
LBFGS Accumulation History: 3 points
Removed measurement 2a611ca to history. Total: 4
Removed measurement 92829d3 to history. Total: 3
Adding measurement 12882124 to history. Total: 3
th(0)=6.708659089542602;dx=-2.6135697596929553E-4
Adding measurement 52717157 to history. Total: 4
New Minimum: 6.708659089542602 > 3.3268686456608094
WOLF (strong): th(33419.43960201202)=3.3268686456608094; dx=5.897234689381631E-5 evalInputDelta=3.3817904438817923
Non-optimal measurement 3.679610560329558 < 3.3268686456608094. Total: 5
END: th(16709.71980100601)=3.679610560329558; dx=-1.0119231453773963E-4 evalInputDelta=3.0290485292130436
Fitness changed from 6.708659089542602 to 3.3268686456608094
Iteration 5 complete. Error: 3.3268686456608094 Total: 5.4460; Orientation: 4.8748; Line Search: 0.4937
Final threshold in iteration 5: 3.3268686456608094 (> 0.0) after 30.391s (< 30.000s)
Returns
3.3268686456608094
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -0.24364495959178212, 0.14223918265636473, -0.30814343769651154 ], [ 0.3552980885468878, -0.06166663572701313, 0.8495060835375332 ], [ 1.1496455639243677, 0.2987256266322955, 0.30578240625702224 ], [ 1.9893862556827417, 0.2742079674790321, 0.22162870764147896 ], [ 0.3592660211526741, -0.9043917163747732, -0.7648548295693876 ], [ 0.23773804714295238, -0.07149036641647649, -1.7443000855624835 ], [ -1.6052585790998086, -0.6903341527200627, 1.3081888701484201 ], [ 2.2818106277601995, -1.733491289600808, 1.4887548449912802 ], ... ],
[ [ 1.0571576721284044, 0.026446863676335752, -1.375460000708813 ], [ -1.3964495546539775, 1.20358367573871, -0.10428116676483662 ], [ -0.4867865736019809, 0.46971323730750847, -1.0138249543385107 ], [ 1.1637467333186353, 1.767540743542392, 1.4380872128464857 ], [ 0.7135716655549663, -0.718023240191052, -0.7822731077847568 ], [ -1.3303598759825128, 1.2575447256408858, 1.288524821161031 ], [ -0.15746386766228826, 1.3377885188651337, -0.7518704058495358 ], [ -1.6339503189192937, -1.435363392004774, -0.6133839833386833 ], ... ],
[ [ 1.0790125699638042, -0.4810731215170583, 1.2423020033790888 ], [ 0.8730046918137506, 1.1209560120082205, 0.5516068851778894 ], [ 0.1023566603718406, 0.7581849503503411, -0.7571898996379596 ], [ 0.22778830019547414, 0.1462879105749164, -2.320003965265831 ], [ -0.06263451835620737, 0.14432668551230554, -1.6486248798470562 ], [ 0.13999951403761857, 0.1480206800673124, 1.382138315816948 ], [ -1.4444781053377238, 1.2799511497787623, -0.14844869425336543 ], [ 0.3754873439813668, -1.4838247706157386, -1.1373264799780562 ], ... ],
[ [ 0.3163061444032052, 0.8713860908389397, -0.1298796169366212 ], [ 0.053999671357952764, -0.33122461042401247, -0.7139943702313944 ], [ 2.3608918417328346, 1.171480116781541, 0.6328775940280356 ], [ 1.685507098977853, 0.6613030289695725, -0.5723463575415222 ], [ 1.8011853920453111, -0.03810837415048149, -1.2915891600701253 ], [ 0.3788139947546951, -0.08648018804331503, -1.2612225528989753 ], [ 0.8865213506607964, -1.4594283071311664, -0.05025487965794134 ], [ -1.1253862185124464, -0.46321721689665524, 1.1490180065472184 ], ... ],
[ [ -2.545850263463219, -1.02979191998828, -0.29150615662507945 ], [ 0.6119018602517329, -0.480332635824924, 0.7793593873702084 ], [ 2.2492295357412186, 1.7105125542169322, -1.0436263622241997 ], [ 0.5329406945344741, -1.4641590072316117, -0.4668330706969899 ], [ 1.180187827406956, 0.6352956083174679, 0.8245037586858095 ], [ -1.1937968709202464, -1.2592474632492716, 0.5481072035780097 ], [ 1.1375521108819497, 0.09024017651867264, 2.0038866974213474 ], [ 1.1524679776269524, 1.931519326237353, -1.8753015229970638 ], ... ],
[ [ 0.5935006527579247, -0.14746449258195063, -0.28550225654575745 ], [ 1.47446113463872, 0.03578589241658048, -0.608352912395663 ], [ 0.5960695361323021, 0.7409132714978883, 0.874091083200383 ], [ -0.30585699854922915, 1.8082075411054244, 1.978677031202564 ], [ 1.3345215345910562, 1.1502480623883653, -0.6190217196569857 ], [ -1.039116335032661, -0.7250517038559531, -0.8221399863831121 ], [ -1.163579647695503, 0.2290418603864341, 0.904831527642257 ], [ 1.2670483681738145, 0.9489690721224662, 0.3325852740186073 ], ... ],
[ [ 1.163028567519055, -1.1764683708471624, -0.22998398957913602 ], [ 1.2090864890305513, 0.43441660788103675, 0.9303674716327845 ], [ -0.8285227690666328, -0.8202940926322384, -0.8949917031931571 ], [ -1.2985803091704844, 0.4067408530775648, 0.5023008435734901 ], [ -1.4359833754353524, 0.11771919997823246, 0.5778045757887056 ], [ -0.9002474768454598, 0.6534599745287242, 0.5923423419719259 ], [ -0.8384540283326163, -1.354076917882386, -1.2466516011764806 ], [ 0.2998772292713932, -1.6253878348270965, -3.3986303093142367 ], ... ],
[ [ -0.16430931034000484, 0.7928752599385133, -1.38258182252798 ], [ -0.9433264543309696, -1.053688394074346, -0.4105769212541742 ], [ -1.211967815126831, 1.7032889310562842, -2.332384668389218 ], [ -1.1205447919420448, -0.5492041227263953, 1.7973627356972344 ], [ -0.5312074199634209, -0.37675154730951066, -0.7311071108860222 ], [ 0.37228450611817854, -0.9146369638341938, 0.5056157253370117 ], [ 1.02129709191584, -1.8884867129669443, 0.7733163630007381 ], [ 0.1389483304901929, 1.3484542595029416, 0.7164134104866945 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.03 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 5.7717488084210045, 0.06096447355581046, -1.2918396068862452 ], [ 1.5670868673651295, 4.88992915509275, 8.207504251964467 ], [ 5.743731061079699, -3.744342515409045, 1.6300931545136346 ], [ -9.312738707456491, 0.8650886960704028, -2.975581539039086 ], [ 7.725772675088442, -7.9775991252802285, -4.550137375419372 ], [ -7.7140948649858085, 1.8351010927483875, 0.7295269600620522 ], [ 8.160947898319444, -1.0926793225630225, -0.7469631146540182 ], [ 1.7069468175106548, 2.1212389779530514, 9.07023887873082 ], ... ],
[ [ -5.210154416887854, 2.210033475239716, -4.199523507150997 ], [ 2.946711579026352, -5.1786189975676145, -5.796607479526361 ], [ 11.623493464008687, 4.288131553632393, -0.47016190991725026 ], [ 6.753060241480269, -5.70679744303315, -3.4482503461899396 ], [ -0.2887922158275081, -0.7328430864264058, -1.856590416952331 ], [ -2.6638138139408345, 1.6277899240455156, -1.5819110952653412 ], [ -9.601771049693301, -16.317653587042177, -4.323744068236552 ], [ 12.725495595046004, 4.057626889324796, 7.044796711472539 ], ... ],
[ [ 1.8245002114492712, 7.5675507088041645, 2.0009609534810804 ], [ -0.9339646007943343, -10.985550422677868, -3.7299174774397486 ], [ -6.96432788495778, -1.8987351234884038, -7.332715736356127 ], [ -2.821221768029979, -2.1202020635258303, -6.974852893925325 ], [ 1.2016786779784796, 8.903826187601204, -4.160752944141974 ], [ -0.391580198753746, -2.521293590457093, 0.4659472202875214 ], [ -7.6588749162121, 3.8281586329775297, -1.8268356652703714 ], [ -5.100517747174451, 9.837443545614606, 10.25415421058175 ], ... ],
[ [ 0.9575812237272624, -8.95871883994408, 1.8076925568511844 ], [ 8.366116784140539, 8.682945703598381, 5.119695371676295 ], [ 4.31904405107702, 3.1912178418052832, -7.4355382630290325 ], [ -5.052757347351781, -1.5729329539552723, -3.035767117395919 ], [ -17.500921348049385, -1.1253320849274562, -9.498885253410757 ], [ -6.955098092493297, -4.695771419043758, 3.415797062454877 ], [ -3.29845973648507, 11.239227084030043, 2.748802301986747 ], [ -3.5716237750306314, -5.089222812353788, -10.640131477384132 ], ... ],
[ [ 1.2247911378162555, 2.9568492276119804, -0.2215994442665713 ], [ 13.884386244370612, 4.62979614295707, 1.7134921931372868 ], [ -4.802179813342435, -7.098310533235523, -6.634586057454412 ], [ 3.9472733972962506, 3.9568872346608637, -2.7291115234989807 ], [ 4.552814640623431, -12.269974245809907, -17.8668604260148 ], [ -0.0602382188009702, -1.8849831370003196, -1.3276624623678877 ], [ 1.802337138247967, 3.221375127936639, 9.186380168556948 ], [ -2.3272404742401664, -0.5824114827674384, -2.2557033655057914 ], ... ],
[ [ -0.06500334015734258, 1.6527775528172886, 3.241193138406586 ], [ -2.0636554630380015, 8.327746113546988, 1.1623453460163289 ], [ 1.2496603483669226, 0.29686834115220845, -9.88534219147389 ], [ 4.25469478130647, -7.3101701976519635, -0.9130660523492495 ], [ -5.309335190732926, -8.288577195792897, 1.099807611900652 ], [ -1.204244558510523, -4.373803917942331, -4.787218909006699 ], [ 18.877444810843688, -0.5072717992297695, 5.256454618147909 ], [ 6.449588133600001, -7.533612820871956, -11.400676376924233 ], ... ],
[ [ 4.347976351552495, 1.9678897822060062, -1.8772894250946797 ], [ 2.060729407202795, -10.00278343929253, -4.633462832506835 ], [ -1.3108346380089162, -13.650060525883731, -8.828985546760762 ], [ 4.822731397645415, -3.090015683246345, -0.7344121940065506 ], [ 3.9663588844613034, 4.266731102982643, -3.299786567056377 ], [ -8.385726613414958, -3.768995861054673, -0.8475797439601276 ], [ 3.689153360825782, -4.1657585070541225, 7.319600948976173 ], [ 0.2015009375428314, 7.441279592360858, 0.019162195111569136 ], ... ],
[ [ 0.2649839481902109, -9.274638472778891, -6.196738504586969 ], [ 11.065425073121187, -0.44442403900735394, 3.8968044410637686 ], [ 3.7419696845860155, -5.8134887864558245, -9.980565829839877 ], [ 2.781234139820054, -0.6382093632968011, 13.090000348303723 ], [ -2.904614520179589, 3.697399705603758, 7.764120329680378 ], [ -2.976616338498635, 6.040418978815328, 9.992724881180504 ], [ 3.111088886435195, 13.158132223787389, 1.6283398743534796 ], [ 2.8297430683892255, 4.938250712592514, -1.7631564694659332 ], ... ],
...
]
TrainingTester.java:432 executed in 0.13 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -1.3565939665073], [52.0, 1.8855992688672973]; valueStats=DoubleSummaryStatistics{count=84, sum=368.484427, min=0.043995, average=4.386719, max=76.842108}
Plotting 52 points for GD
Plotting 27 points for CjGD
Plotting 5 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.02 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -1.3565939665073], [28.769, 1.8855992688672973]; valueStats=DoubleSummaryStatistics{count=84, sum=368.484427, min=0.043995, average=4.386719, max=76.842108}
Plotting 52 points for GD
Plotting 27 points for CjGD
Plotting 5 points for LBFGS
Returns
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:370 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0042 = network_target.state();
assert temp_18_0042 != null;
String temp_18_0041 = temp_18_0042.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0042.freeRef();
return temp_18_0041;
Returns
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 8.77 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13561404893284
Reset training subject: 13561558954903
Constructing line search parameters: GD
th(0)=89.97905736583994;dx=-159.47279177709362
New Minimum: 89.97905736583994 > 74.38853294912732
WOLF (strong): th(2.154434690031884)=74.38853294912732; dx=144.99983096853725 evalInputDelta=15.590524416712611
New Minimum: 74.38853294912732 > 0.18799758146127732
END: th(1.077217345015942)=0.18799758146127732; dx=-7.236480404278069 evalInputDelta=89.79105978437866
Fitness changed from 89.97905736583994 to 0.18799758146127732
Iteration 1 complete. Error: 0.18799758146127732 Total: 0.7707; Orientation: 0.0004; Line Search: 0.3485
th(0)=0.18799758146127732;dx=-0.33278059055385106
Armijo: th(2.3207944168063896)=0.2088938910037837; dx=0.3507884842226584 evalInputDelta=-0.020896309542506375
New Minimum: 0.18799758146127732 > 1.428247021764748E-4
WOLF (strong): th(1.1603972084031948)=1.428247021764748E-4; dx=0.00900394683440401 evalInputDelta=0.18785475675910085
END: th(0.3867990694677316)=0.08131201553483167; dx=-0.21885241142443268 evalInputDelta=0.10668556592644565
Fitness changed from 0.18799758146127732 to 1.428247021764748E-4
Iteration 2 complete. Error: 1.428247021764748E-4 Total: 0.5429; Orientation: 0.0004; Line Search: 0.4343
th(0)=1.428247021764748E-4;dx=-2.533554913456661E-4
New Minimum: 1.428247021764748E-4 > 9.722619531623068E-6
END: th(0.8333333333333335)=9.722619531623068E-6; dx=-6.608950700197737E-5 evalInputDelta=1.3310208264485172E-4
Fitness changed from 1.428247021764748E-4 to 9.722619531623068E-6
Iteration 3 complete. Error: 9.722619531623068E-6 Total: 0.3245; Orientation: 0.0004; Line Search: 0.2163
th(0)=9.722619531623068E-6;dx=-1.72439852646261E-5
New Minimum: 9.722619531623068E-6 > 3.4095640165191417E-6
WOLF (strong): th(1.7953622416932369)=3.4095640165191417E-6; dx=1.0211359348253925E-5 evalInputDelta=6.313055515103926E-6
New Minimum: 3.4095640165191417E-6 > 4.0455564274107467E-7
END: th(0.8976811208466184)=4.0455564274107467E-7; dx=-3.5163129581863272E-6 evalInputDelta=9.318063888881993E-6
Fitness changed from 9.722619531623068E-6 to 4.0455564274107467E-7
Iteration 4 complete. Error: 4.0455564274107467E-7 Total: 0.4323; Orientation: 0.0004; Line Search: 0.3247
th(0)=4.0455564274107467E-7;dx=-7.173525926969966E-7
New Minimum: 4.0455564274107467E-7 > 2.0666145418321104E-7
WOLF (strong): th(1.9339953473386584)=2.0666145418321104E-7; dx=5.127045424003029E-7 evalInputDelta=1.9789418855786363E-7
New Minimum: 2.0666145418321104E-7 > 8.242951432291775E-9
END: th(0.9669976736693292)=8.242951432291775E-9; dx=-1.0232402514839722E-7 evalInputDelta=3.963126913087829E-7
Fitness changed from 4.0455564274107467E-7 to 8.242951432291775E-9
Iteration 5 complete. Error: 8.242951432291775E-9 Total: 0.4321; Orientation: 0.0003; Line Search: 0.3241
th(0)=8.242951432291775E-9;dx=-1.4611136826711225E-8
New Minimum: 8.242951432291775E-9 > 5.906263096286987E-9
WOLF (strong): th(2.083333333333334)=5.906263096286987E-9; dx=1.2367916024146072E-8 evalInputDelta=2.3366883360047885E-9
New Minimum: 5.906263096286987E-9 > 4.8812251046269134E-11
END: th(1.041666666666667)=4.8812251046269134E-11; dx=-1.1216104012762122E-9 evalInputDelta=8.194139181245506E-9
Fitness changed from 8.242951432291775E-9 to 4.8812251046269134E-11
Iteration 6 complete. Error: 4.8812251046269134E-11 Total: 0.4330; Orientation: 0.0003; Line Search: 0.3249
Low gradient: 9.298530771514791E-6
th(0)=4.8812251046269134E-11;dx=-8.646267450880746E-11
New Minimum: 4.8812251046269134E-11 > 4.761593881610975E-11
WOLF (strong): th(2.244202802116546)=4.761593881610975E-11; dx=8.539653892692437E-11 evalInputDelta=1.1963122301593834E-12
New Minimum: 4.761593881610975E-11 > 3.2288854997063547E-15
END: th(1.122101401058273)=3.2288854997063547E-15; dx=-5.330677922374132E-13 evalInputDelta=4.880902216076943E-11
Fitness changed from 4.8812251046269134E-11 to 3.2288854997063547E-15
Iteration 7 complete. Error: 3.2288854997063547E-15 Total: 0.4326; Orientation: 0.0005; Line Search: 0.3246
Low gradient: 7.545674897831053E-8
th(0)=3.2288854997063547E-15;dx=-5.6937209663757675E-15
Armijo: th(2.417494184173323)=4.133834018336776E-15; dx=6.44238751891365E-15 evalInputDelta=-9.049485186304216E-16
New Minimum: 3.2288854997063547E-15 > 1.3988297057592668E-17
WOLF (strong): th(1.2087470920866614)=1.3988297057592668E-17; dx=3.7433326986495866E-16 evalInputDelta=3.214897202648762E-15
END: th(0.4029156973622205)=1.3422816658589337E-15; dx=-3.671036222628186E-15 evalInputDelta=1.886603833847421E-15
Fitness changed from 3.2288854997063547E-15 to 1.3988297057592668E-17
Iteration 8 complete. Error: 1.3988297057592668E-17 Total: 0.5415; Orientation: 0.0005; Line Search: 0.4333
Low gradient: 4.9673820407967506E-9
th(0)=1.3988297057592668E-17;dx=-2.4674884339230092E-17
New Minimum: 1.3988297057592668E-17 > 7.685965830347263E-19
END: th(0.8680555555555558)=7.685965830347263E-19; dx=-5.783306071497025E-18 evalInputDelta=1.3219700474557943E-17
Fitness changed from 1.3988297057592668E-17 to 7.685965830347263E-19
Iteration 9 complete. Error: 7.685965830347263E-19 Total: 0.3240; Orientation: 0.0004; Line Search: 0.2161
Low gradient: 1.1643329945774956E-9
th(0)=7.685965830347263E-19;dx=-1.3556713222617984E-18
New Minimum: 7.685965830347263E-19 > 3.240845120484099E-19
WOLF (strong): th(1.8701690017637886)=3.240845120484099E-19; dx=8.803003042439817E-19 evalInputDelta=4.445120709863164E-19
New Minimum: 3.240845120484099E-19 > 2.3634952425388987E-20
END: th(0.9350845008818943)=2.3634952425388987E-20; dx=-2.3768554591713575E-19 evalInputDelta=7.4496163060933725E-19
Fitness changed from 7.685965830347263E-19 to 2.3634952425388987E-20
Iteration 10 complete. Error: 2.3634952425388987E-20 Total: 0.4310; Orientation: 0.0004; Line Search: 0.3230
Low gradient: 2.0416531437303222E-10
th(0)=2.3634952425388987E-20;dx=-4.168347559303907E-20
New Minimum: 2.3634952425388987E-20 > 1.425127787233195E-20
WOLF (strong): th(2.0145784868111027)=1.425127787233195E-20; dx=3.236770512061463E-20 evalInputDelta=9.383674553057036E-21
New Minimum: 1.425127787233195E-20 > 2.953744573168064E-22
END: th(1.0072892434055514)=2.953744573168064E-22; dx=-4.657862895699307E-21 evalInputDelta=2.333957796807218E-20
Fitness changed from 2.3634952425388987E-20 to 2.953744573168064E-22
Iteration 11 complete. Error: 2.953744573168064E-22 Total: 0.4307; Orientation: 0.0003; Line Search: 0.3234
Zero gradient: 2.28220522101728E-11
th(0)=2.953744573168064E-22;dx=-5.208460670838532E-22
New Minimum: 2.953744573168064E-22 > 2.4641270246310995E-22
WOLF (strong): th(2.1701388888888897)=2.4641270246310995E-22; dx=4.757228898087843E-22 evalInputDelta=4.896175485369645E-23
New Minimum: 2.4641270246310995E-22 > 5.57065415325542E-25
END: th(1.0850694444444449)=5.57065415325542E-25; dx=-2.2561282341539242E-23 evalInputDelta=2.9481739190148087E-22
Fitness changed from 2.953744573168064E-22 to 5.57065415325542E-25
Iteration 12 complete. Error: 5.57065415325542E-25 Total: 0.4308; Orientation: 0.0004; Line Search: 0.3234
Zero gradient: 9.909121051928188E-13
th(0)=5.57065415325542E-25;dx=-9.819068002176643E-25
Armijo: th(2.337711252204736)=6.263277994032249E-25; dx=1.0411619313069124E-24 evalInputDelta=-6.926238407768291E-26
New Minimum: 5.57065415325542E-25 > 5.123019368115922E-28
WOLF (strong): th(1.168855626102368)=5.123019368115922E-28; dx=2.957412684317988E-26 evalInputDelta=5.565531133887304E-25
END: th(0.38961854203412266)=2.4017181206621804E-25; dx=-6.447251045124196E-25 evalInputDelta=3.1689360325932396E-25
Fitness changed from 5.57065415325542E-25 to 5.123019368115922E-28
Iteration 13 complete. Error: 5.123019368115922E-28 Total: 0.5409; Orientation: 0.0004; Line Search: 0.4334
Zero gradient: 2.997018221427835E-14
th(0)=5.123019368115922E-28;dx=-8.982118219570463E-28
New Minimum: 5.123019368115922E-28 > 3.62620538848139E-29
END: th(0.8394077028379595)=3.62620538848139E-29; dx=-2.3000898745335644E-28 evalInputDelta=4.760398829267783E-28
Fitness changed from 5.123019368115922E-28 to 3.62620538848139E-29
Iteration 14 complete. Error: 3.62620538848139E-29 Total: 0.3246; Orientation: 0.0005; Line Search: 0.2166
Zero gradient: 7.68392439233579E-15
th(0)=3.62620538848139E-29;dx=-5.904269406713294E-29
New Minimum: 3.62620538848139E-29 > 1.4420348670271094E-29
WOLF (strong): th(1.8084490740740748)=1.4420348670271094E-29; dx=3.495216890219964E-29 evalInputDelta=2.1841705214542803E-29
New Minimum: 1.4420348670271094E-29 > 3.0788825188895786E-30
END: th(0.9042245370370374)=3.0788825188895786E-30; dx=-1.1454628920707434E-29 evalInputDelta=3.318317136592432E-29
Fitness changed from 3.62620538848139E-29 to 3.0788825188895786E-30
Iteration 15 complete. Error: 3.0788825188895786E-30 Total: 0.4336; Orientation: 0.0005; Line Search: 0.3261
Zero gradient: 1.5344099433250418E-15
th(0)=3.0788825188895786E-30;dx=-2.3544138741747584E-30
New Minimum: 3.0788825188895786E-30 > 2.3222254434184217E-30
WOLF (strong): th(1.9480927101706134)=2.3222254434184217E-30; dx=1.7065634855724564E-30 evalInputDelta=7.566570754711569E-31
New Minimum: 2.3222254434184217E-30 > 1.5926440279937047E-31
END: th(0.9740463550853067)=1.5926440279937047E-31; dx=-8.422490552359147E-32 evalInputDelta=2.9196181160902082E-30
Fitness changed from 3.0788825188895786E-30 to 1.5926440279937047E-31
Iteration 16 complete. Error: 1.5926440279937047E-31 Total: 0.4334; Orientation: 0.0005; Line Search: 0.3253
Zero gradient: 1.297260598015477E-16
th(0)=1.5926440279937047E-31;dx=-1.6828850591634733E-32
New Minimum: 1.5926440279937047E-31 > 1.4845446052068562E-31
WOLF (strong): th(2.0985192570948987)=1.4845446052068562E-31; dx=1.5819767764883198E-32 evalInputDelta=1.0809942278684853E-32
New Minimum: 1.4845446052068562E-31 > 4.054328623983581E-34
END: th(1.0492596285474494)=4.054328623983581E-34; dx=-1.5324063496288922E-35 evalInputDelta=1.5885896993697212E-31
Fitness changed from 1.5926440279937047E-31 to 4.054328623983581E-34
Iteration 17 complete. Error: 4.054328623983581E-34 Total: 0.4321; Orientation: 0.0004; Line Search: 0.3243
Zero gradient: 2.0475554784874333E-18
th(0)=4.054328623983581E-34;dx=-4.192483437483903E-36
Armijo: th(2.2605613425925934)=1.8581022582828517E-33; dx=5.874422239587826E-36 evalInputDelta=-1.4526693958844936E-33
New Minimum: 4.054328623983581E-34 > 1.6379178208149268E-34
END: th(1.1302806712962967)=1.6379178208149268E-34; dx=-2.0177886743866465E-36 evalInputDelta=2.416410803168654E-34
Fitness changed from 4.054328623983581E-34 to 1.6379178208149268E-34
Iteration 18 complete. Error: 1.6379178208149268E-34 Total: 0.4348; Orientation: 0.0006; Line Search: 0.3266
Zero gradient: 9.891174974707905E-19
th(0)=1.6379178208149268E-34;dx=-9.783534238028796E-37
New Minimum: 1.6379178208149268E-34 > 0.0
END: th(2.4351158877132666)=0.0; dx=0.0 evalInputDelta=1.6379178208149268E-34
Fitness changed from 1.6379178208149268E-34 to 0.0
Iteration 19 complete. Error: 0.0 Total: 0.3244; Orientation: 0.0005; Line Search: 0.2167
Zero gradient: 0.0
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.3224; Orientation: 0.0004; Line Search: 0.2146
Iteration 20 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 20
Final threshold in iteration 20: 0.0 (> 0.0) after 8.773s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 5.95 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13570199870121
Reset training subject: 13570307203174
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=89.97905736583994}, derivative=-159.47279177709362}
New Minimum: 89.97905736583994 > 89.97905734989266
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=89.97905734989266}, derivative=-159.47279176296163}, evalInputDelta = -1.5947279052852537E-8
New Minimum: 89.97905734989266 > 89.97905725420898
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=89.97905725420898}, derivative=-159.47279167816657}, evalInputDelta = -1.1163095336996776E-7
New Minimum: 89.97905725420898 > 89.97905658442325
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=89.97905658442325}, derivative=-159.47279108460742}, evalInputDelta = -7.81416687800629E-7
New Minimum: 89.97905658442325 > 89.97905189592325
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=89.97905189592325}, derivative=-159.47278692969144}, evalInputDelta = -5.469916686706711E-6
New Minimum: 89.97905189592325 > 89.97901907642671
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=89.97901907642671}, derivative=-159.47275784527983}, evalInputDelta = -3.8289413225811586E-5
New Minimum: 89.97901907642671 > 89.9787893401184
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=89.9787893401184}, derivative=-159.47255425439658}, evalInputDelta = -2.6802572153883375E-4
New Minimum: 89.9787893401184 > 89.97718119417247
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=89.97718119417247}, derivative=-159.47112911821822}, evalInputDelta = -0.00187617166746179
New Minimum: 89.97718119417247 > 89.96592457494896
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=89.96592457494896}, derivative=-159.4611531649644}, evalInputDelta = -0.013132790890978185
New Minimum: 89.96592457494896 > 89.887147957888
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=89.887147957888}, derivative=-159.39132149218793}, evalInputDelta = -0.09190940795193114
New Minimum: 89.887147957888 > 89.33667779613411
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=89.33667779613411}, derivative=-158.9024997827541}, evalInputDelta = -0.6423795697058239
New Minimum: 89.33667779613411 > 85.53072838983033
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=85.53072838983033}, derivative=-155.48074781671318}, evalInputDelta = -4.448328976009606
New Minimum: 85.53072838983033 > 61.20882711840018
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=61.20882711840018}, derivative=-131.52848405443197}, evalInputDelta = -28.770230247439756
New Minimum: 61.20882711840018 > 4.623002280507425
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=4.623002280507425}, derivative=36.13736228153972}, evalInputDelta = -85.35605508533251
4.623002280507425 <= 89.97905736583994
New Minimum: 4.623002280507425 > 0.0027255471210762324
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=0.0027255471210762324}, derivative=-8.09213807073661E-14}, evalInputDelta = -89.97633181871886
Left bracket at 1.1284223573947967
Converged to left
Fitness changed from 89.97905736583994 to 0.0027255471210762324
Iteration 1 complete. Error: 0.0027255471210762324 Total: 2.0559; Orientation: 0.0004; Line Search: 1.7343
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0027255471210762324}, derivative=-0.004835958619481105}
New Minimum: 0.0027255471210762324 > 1.4995035676164055E-7
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=1.4995035676164055E-7}, derivative=5.502801481132531E-6}, evalInputDelta = -0.0027253971707194707
1.4995035676164055E-7 <= 0.0027255471210762324
Converged to right
Fitness changed from 0.0027255471210762324 to 1.4995035676164055E-7
Iteration 2 complete. Error: 1.4995035676164055E-7 Total: 0.3249; Orientation: 0.0003; Line Search: 0.2170
F(0.0) = LineSearchPoint{point=PointSample{avg=1.4995035676164055E-7}, derivative=-2.6656138389265146E-7}
New Minimum: 1.4995035676164055E-7 > 1.0494453528277492E-11
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=1.0494453528277492E-11}, derivative=8.100695456234285E-10}, evalInputDelta = -1.4993986230811226E-7
1.0494453528277492E-11 <= 1.4995035676164055E-7
Converged to right
Fitness changed from 1.4995035676164055E-7 to 1.0494453528277492E-11
Iteration 3 complete. Error: 1.0494453528277492E-11 Total: 0.3240; Orientation: 0.0004; Line Search: 0.2164
Low gradient: 4.323121669580061E-6
F(0.0) = LineSearchPoint{point=PointSample{avg=1.0494453528277492E-11}, derivative=-1.8689380969992696E-11}
New Minimum: 1.0494453528277492E-11 > 8.356700785140581E-16
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=8.356700785140581E-16}, derivative=9.063948043439947E-14}, evalInputDelta = -1.0493617858198977E-11
8.356700785140581E-16 <= 1.0494453528277492E-11
Converged to right
Fitness changed from 1.0494453528277492E-11 to 8.356700785140581E-16
Iteration 4 complete. Error: 8.356700785140581E-16 Total: 0.3234; Orientation: 0.0004; Line Search: 0.2155
Low gradient: 3.860388650020554E-8
F(0.0) = LineSearchPoint{point=PointSample{avg=8.356700785140581E-16}, derivative=-1.4902600529207517E-15}
New Minimum: 8.356700785140581E-16 > 7.181267731301299E-20
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=7.181267731301299E-20}, derivative=9.257376928533666E-18}, evalInputDelta = -8.355982658367451E-16
7.181267731301299E-20 <= 8.356700785140581E-16
Converged to right
Fitness changed from 8.356700785140581E-16 to 7.181267731301299E-20
Iteration 5 complete. Error: 7.181267731301299E-20 Total: 0.3231; Orientation: 0.0004; Line Search: 0.2157
Low gradient: 3.580292841138359E-10
F(0.0) = LineSearchPoint{point=PointSample{avg=7.181267731301299E-20}, derivative=-1.2818496828306583E-19}
New Minimum: 7.181267731301299E-20 > 6.4837492752315496E-24
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=6.4837492752315496E-24}, derivative=9.166859303497028E-22}, evalInputDelta = -7.180619356373776E-20
6.4837492752315496E-24 <= 7.181267731301299E-20
Converged to right
Fitness changed from 7.181267731301299E-20 to 6.4837492752315496E-24
Iteration 6 complete. Error: 6.4837492752315496E-24 Total: 0.3247; Orientation: 0.0004; Line Search: 0.2169
Zero gradient: 3.403068720958618E-12
F(0.0) = LineSearchPoint{point=PointSample{avg=6.4837492752315496E-24}, derivative=-1.1580876719566925E-23}
New Minimum: 6.4837492752315496E-24 > 6.096850720059554E-28
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=6.096850720059554E-28}, derivative=9.030007637227354E-26}, evalInputDelta = -6.483139590159544E-24
6.096850720059554E-28 <= 6.4837492752315496E-24
Converged to right
Fitness changed from 6.4837492752315496E-24 to 6.096850720059554E-28
Iteration 7 complete. Error: 6.096850720059554E-28 Total: 0.3235; Orientation: 0.0004; Line Search: 0.2159
Zero gradient: 3.2926432155212103E-14
F(0.0) = LineSearchPoint{point=PointSample{avg=6.096850720059554E-28}, derivative=-1.0841499344717855E-27}
New Minimum: 6.096850720059554E-28 > 4.408749276980764E-31
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=4.408749276980764E-31}, derivative=5.1304713575154845E-30}, evalInputDelta = -6.092441970782573E-28
4.408749276980764E-31 <= 6.096850720059554E-28
Converged to right
Fitness changed from 6.096850720059554E-28 to 4.408749276980764E-31
Iteration 8 complete. Error: 4.408749276980764E-31 Total: 0.3235; Orientation: 0.0004; Line Search: 0.2155
Zero gradient: 2.782209764810231E-16
F(0.0) = LineSearchPoint{point=PointSample{avg=4.408749276980764E-31}, derivative=-7.740691175405399E-32}
New Minimum: 4.408749276980764E-31 > 9.586965407909465E-34
F(1.1284223573947967) = LineSearchPoint{point=PointSample{avg=9.586965407909465E-34}, derivative=-2.765003118883209E-35}, evalInputDelta = -4.399162311572854E-31
F(7.8989565017635766) = LineSearchPoint{point=PointSample{avg=2.6423973685983704E-30}, derivative=4.635859561784306E-31}, evalInputDelta = 2.201522440900294E-30
F(0.6076120385971981) = LineSearchPoint{point=PointSample{avg=8.752267456737397E-32}, derivative=-1.412151085846668E-32}, evalInputDelta = -3.5335225313070237E-31
F(4.253284270180387) = LineSearchPoint{point=PointSample{avg=1.16395278018708E-30}, derivative=2.254829533988194E-31}, evalInputDelta = 7.2307785248900365E-31
F(0.327175713090799) = LineSearchPoint{point=PointSample{avg=4.145614975286775E-31}, derivative=-7.18871256107655E-32}, evalInputDelta = -2.631343016939883E-32
F(2.2902299916355933) = LineSearchPoint{point=PointSample{avg=4.277737420549393E-31}, derivative=7.682016394301976E-32}, evalInputDelta = -1.31011856431371E-32
4.277737420549393E-31 <= 4.408749276980764E-31
F(1.1494715182645963) = LineSearchPoint{point=PointSample{avg=9.586965407909465E-34}, derivative=-2.765003118883209E-35}, evalInputDelta = -4.399162311572854E-31
Left bracket at 1.1494715182645963
Converged to left
Fitness changed from 4.408749276980764E-31 to 9.586965407909465E-34
Iteration 9 complete. Error: 9.586965407909465E-34 Total: 1.0835; Orientation: 0.0004; Line Search: 0.9757
Zero gradient: 2.9061804413422327E-18
F(0.0) = LineSearchPoint{point=PointSample{avg=9.586965407909465E-34}, derivative=-8.445884757640134E-36}
New Minimum: 9.586965407909465E-34 > 0.0
F(1.1494715182645963) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -9.586965407909465E-34
0.0 <= 9.586965407909465E-34
Converged to right
Fitness changed from 9.586965407909465E-34 to 0.0
Iteration 10 complete. Error: 0.0 Total: 0.3242; Orientation: 0.0005; Line Search: 0.2164
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.2161; Orientation: 0.0004; Line Search: 0.1076
Iteration 11 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 11
Final threshold in iteration 11: 0.0 (> 0.0) after 5.947s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 10.13 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13576168245588
Reset training subject: 13576275430455
Adding measurement 782dbaa2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 89.97905736583994 < 89.97905736583994. Total: 1
th(0)=89.97905736583994;dx=-159.47279177709362
Adding measurement 24a18d4e to history. Total: 1
New Minimum: 89.97905736583994 > 74.38853294912732
WOLF (strong): th(2.154434690031884)=74.38853294912732; dx=144.99983096853725 evalInputDelta=15.590524416712611
Adding measurement 4c1f5c4e to history. Total: 2
New Minimum: 74.38853294912732 > 0.18799758146127732
END: th(1.077217345015942)=0.18799758146127732; dx=-7.236480404278069 evalInputDelta=89.79105978437866
Fitness changed from 89.97905736583994 to 0.18799758146127732
Iteration 1 complete. Error: 0.18799758146127732 Total: 0.6496; Orientation: 0.0011; Line Search: 0.3265
Non-optimal measurement 0.18799758146127732 < 0.18799758146127732. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 0.18799758146127732 < 0.18799758146127732. Total: 3
th(0)=0.18799758146127732;dx=-0.33278059055385106
Non-optimal measurement 0.2088938910037837 < 0.18799758146127732. Total: 3
Armijo: th(2.3207944168063896)=0.2088938910037837; dx=0.3507884842226584 evalInputDelta=-0.020896309542506375
Adding measurement 6a37d9d6 to history. Total: 3
New Minimum: 0.18799758146127732 > 1.428247021764748E-4
WOLF (strong): th(1.1603972084031948)=1.428247021764748E-4; dx=0.00900394683440401 evalInputDelta=0.18785475675910085
Non-optimal measurement 0.08131201553483167 < 1.428247021764748E-4. Total: 4
END: th(0.3867990694677316)=0.08131201553483167; dx=-0.21885241142443268 evalInputDelta=0.10668556592644565
Fitness changed from 0.18799758146127732 to 1.428247021764748E-4
Iteration 2 complete. Error: 1.428247021764748E-4 Total: 0.5419; Orientation: 0.0009; Line Search: 0.4341
Non-optimal measurement 1.428247021764748E-4 < 1.428247021764748E-4. Total: 4
Rejected: LBFGS Orientation magnitude: 1.795e-02, gradient 1.592e-02, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.428247021764748E-4, 0.18799758146127732, 74.38853294912732, 89.97905736583994
LBFGS Accumulation History: 3 points
Removed measurement 6a37d9d6 to history. Total: 3
Adding measurement 78a85ce9 to history. Total: 3
th(0)=1.428247021764748E-4;dx=-2.533554913456661E-4
Adding measurement 12d7d252 to history. Total: 4
New Minimum: 1.428247021764748E-4 > 9.722619531623068E-6
END: th(0.8333333333333335)=9.722619531623068E-6; dx=-6.608950700197737E-5 evalInputDelta=1.3310208264485172E-4
Fitness changed from 1.428247021764748E-4 to 9.722619531623068E-6
Iteration 3 complete. Error: 9.722619531623068E-6 Total: 0.3463; Orientation: 0.0212; Line Search: 0.2176
Non-optimal measurement 9.722619531623068E-6 < 9.722619531623068E-6. Total: 5
Rejected: LBFGS Orientation magnitude: 4.683e-03, gradient 4.153e-03, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 9.722619531623068E-6, 1.428247021764748E-4, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 4.683e-03, gradient 4.153e-03, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 9.722619531623068E-6, 1.428247021764748E-4, 0.18799758146127732, 74.38853294912732
LBFGS Accumulation History: 3 points
Removed measurement 12d7d252 to history. Total: 4
Removed measurement 78a85ce9 to history. Total: 3
Adding measurement 2532f4f6 to history. Total: 3
th(0)=9.722619531623068E-6;dx=-1.72439852646261E-5
Adding measurement b3b6146 to history. Total: 4
New Minimum: 9.722619531623068E-6 > 3.4095640165191417E-6
WOLF (strong): th(1.7953622416932369)=3.4095640165191417E-6; dx=1.0211359348253925E-5 evalInputDelta=6.313055515103926E-6
Adding measurement 64762427 to history. Total: 5
New Minimum: 3.4095640165191417E-6 > 4.0455564274107467E-7
END: th(0.8976811208466184)=4.0455564274107467E-7; dx=-3.5163129581863272E-6 evalInputDelta=9.318063888881993E-6
Fitness changed from 9.722619531623068E-6 to 4.0455564274107467E-7
Iteration 4 complete. Error: 4.0455564274107467E-7 Total: 0.4809; Orientation: 0.0483; Line Search: 0.3252
Non-optimal measurement 4.0455564274107467E-7 < 4.0455564274107467E-7. Total: 6
Rejected: LBFGS Orientation magnitude: 9.553e-04, gradient 8.470e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.0455564274107467E-7, 3.4095640165191417E-6, 9.722619531623068E-6, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 9.553e-04, gradient 8.470e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.0455564274107467E-7, 3.4095640165191417E-6, 9.722619531623068E-6, 0.18799758146127732, 74.38853294912732
Rejected: LBFGS Orientation magnitude: 9.553e-04, gradient 8.470e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.0455564274107467E-7, 3.4095640165191417E-6, 9.722619531623068E-6, 0.18799758146127732
LBFGS Accumulation History: 3 points
Removed measurement 64762427 to history. Total: 5
Removed measurement b3b6146 to history. Total: 4
Removed measurement 2532f4f6 to history. Total: 3
Adding measurement 48e94a3b to history. Total: 3
th(0)=4.0455564274107467E-7;dx=-7.173525926969966E-7
Adding measurement 4079c3e0 to history. Total: 4
New Minimum: 4.0455564274107467E-7 > 2.0666145418321104E-7
WOLF (strong): th(1.9339953473386584)=2.0666145418321104E-7; dx=5.127045424003029E-7 evalInputDelta=1.9789418855786363E-7
Adding measurement 379ed0a7 to history. Total: 5
New Minimum: 2.0666145418321104E-7 > 8.242951432291775E-9
END: th(0.9669976736693292)=8.242951432291775E-9; dx=-1.0232402514839722E-7 evalInputDelta=3.963126913087829E-7
Fitness changed from 4.0455564274107467E-7 to 8.242951432291775E-9
Iteration 5 complete. Error: 8.242951432291775E-9 Total: 0.5138; Orientation: 0.0794; Line Search: 0.3268
Non-optimal measurement 8.242951432291775E-9 < 8.242951432291775E-9. Total: 6
Rejected: LBFGS Orientation magnitude: 1.364e-04, gradient 1.209e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.242951432291775E-9, 2.0666145418321104E-7, 4.0455564274107467E-7, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 1.364e-04, gradient 1.209e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.242951432291775E-9, 2.0666145418321104E-7, 4.0455564274107467E-7, 0.18799758146127732, 74.38853294912732
Rejected: LBFGS Orientation magnitude: 1.364e-04, gradient 1.209e-04, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.242951432291775E-9, 2.0666145418321104E-7, 4.0455564274107467E-7, 0.18799758146127732
LBFGS Accumulation History: 3 points
Removed measurement 379ed0a7 to history. Total: 5
Removed measurement 4079c3e0 to history. Total: 4
Removed measurement 48e94a3b to history. Total: 3
Adding measurement 714a5f8b to history. Total: 3
th(0)=8.242951432291775E-9;dx=-1.4611136826711225E-8
Adding measurement 3514342b to history. Total: 4
New Minimum: 8.242951432291775E-9 > 5.906263096286987E-9
WOLF (strong): th(2.083333333333334)=5.906263096286987E-9; dx=1.2367916024146072E-8 evalInputDelta=2.3366883360047885E-9
Adding measurement 78cdd8c2 to history. Total: 5
New Minimum: 5.906263096286987E-9 > 4.8812251046269134E-11
END: th(1.041666666666667)=4.8812251046269134E-11; dx=-1.1216104012762122E-9 evalInputDelta=8.194139181245506E-9
Fitness changed from 8.242951432291775E-9 to 4.8812251046269134E-11
Iteration 6 complete. Error: 4.8812251046269134E-11 Total: 0.5142; Orientation: 0.0818; Line Search: 0.3250
Non-optimal measurement 4.8812251046269134E-11 < 4.8812251046269134E-11. Total: 6
Rejected
...skipping 15179 bytes...
92432E-29
Fitness changed from 3.62620538848139E-29 to 3.0788825188895786E-30
Iteration 15 complete. Error: 3.0788825188895786E-30 Total: 0.5925; Orientation: 0.1597; Line Search: 0.3255
Non-optimal measurement 3.0788825188895786E-30 < 3.0788825188895786E-30. Total: 6
Rejected: LBFGS Orientation magnitude: 1.742e-15, gradient 1.534e-15, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.0788825188895786E-30, 1.4420348670271094E-29, 3.62620538848139E-29, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 1.742e-15, gradient 1.534e-15, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.0788825188895786E-30, 1.4420348670271094E-29, 3.62620538848139E-29, 0.18799758146127732, 74.38853294912732
Rejected: LBFGS Orientation magnitude: 1.742e-15, gradient 1.534e-15, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.0788825188895786E-30, 1.4420348670271094E-29, 3.62620538848139E-29, 0.18799758146127732
LBFGS Accumulation History: 3 points
Removed measurement 5d162c2d to history. Total: 5
Removed measurement 5eb69158 to history. Total: 4
Removed measurement 6383f7eb to history. Total: 3
Adding measurement 6a718d0 to history. Total: 3
th(0)=3.0788825188895786E-30;dx=-2.3544138741747584E-30
Adding measurement 740fa786 to history. Total: 4
New Minimum: 3.0788825188895786E-30 > 2.3222254434184217E-30
WOLF (strong): th(1.9480927101706134)=2.3222254434184217E-30; dx=1.7065634855724564E-30 evalInputDelta=7.566570754711569E-31
Adding measurement 320fb782 to history. Total: 5
New Minimum: 2.3222254434184217E-30 > 1.5926440279937047E-31
END: th(0.9740463550853067)=1.5926440279937047E-31; dx=-8.422490552359147E-32 evalInputDelta=2.9196181160902082E-30
Fitness changed from 3.0788825188895786E-30 to 1.5926440279937047E-31
Iteration 16 complete. Error: 1.5926440279937047E-31 Total: 0.5164; Orientation: 0.0845; Line Search: 0.3245
Non-optimal measurement 1.5926440279937047E-31 < 1.5926440279937047E-31. Total: 6
Rejected: LBFGS Orientation magnitude: 1.460e-16, gradient 1.297e-16, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.5926440279937047E-31, 2.3222254434184217E-30, 3.0788825188895786E-30, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 1.460e-16, gradient 1.297e-16, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.5926440279937047E-31, 2.3222254434184217E-30, 3.0788825188895786E-30, 0.18799758146127732, 74.38853294912732
Rejected: LBFGS Orientation magnitude: 1.462e-16, gradient 1.297e-16, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.5926440279937047E-31, 2.3222254434184217E-30, 3.0788825188895786E-30, 0.18799758146127732
LBFGS Accumulation History: 3 points
Removed measurement 320fb782 to history. Total: 5
Removed measurement 740fa786 to history. Total: 4
Removed measurement 6a718d0 to history. Total: 3
Adding measurement 3f5c9a6 to history. Total: 3
th(0)=1.5926440279937047E-31;dx=-1.6828850591634733E-32
Adding measurement 27c9b0ae to history. Total: 4
New Minimum: 1.5926440279937047E-31 > 1.4845446052068562E-31
WOLF (strong): th(2.0985192570948987)=1.4845446052068562E-31; dx=1.5819767764883198E-32 evalInputDelta=1.0809942278684853E-32
Adding measurement 2c122918 to history. Total: 5
New Minimum: 1.4845446052068562E-31 > 4.054328623983581E-34
END: th(1.0492596285474494)=4.054328623983581E-34; dx=-1.5324063496288922E-35 evalInputDelta=1.5885896993697212E-31
Fitness changed from 1.5926440279937047E-31 to 4.054328623983581E-34
Iteration 17 complete. Error: 4.054328623983581E-34 Total: 0.5100; Orientation: 0.0772; Line Search: 0.3247
Non-optimal measurement 4.054328623983581E-34 < 4.054328623983581E-34. Total: 6
Rejected: LBFGS Orientation magnitude: 2.312e-18, gradient 2.048e-18, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.054328623983581E-34, 1.4845446052068562E-31, 1.5926440279937047E-31, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 2.312e-18, gradient 2.048e-18, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.054328623983581E-34, 1.4845446052068562E-31, 1.5926440279937047E-31, 0.18799758146127732, 74.38853294912732
Rejected: LBFGS Orientation magnitude: 2.315e-18, gradient 2.048e-18, dot -1.000; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.054328623983581E-34, 1.4845446052068562E-31, 1.5926440279937047E-31, 0.18799758146127732
LBFGS Accumulation History: 3 points
Removed measurement 2c122918 to history. Total: 5
Removed measurement 27c9b0ae to history. Total: 4
Removed measurement 3f5c9a6 to history. Total: 3
Adding measurement 28e720af to history. Total: 3
th(0)=4.054328623983581E-34;dx=-4.192483437483903E-36
Non-optimal measurement 1.8581022582828517E-33 < 4.054328623983581E-34. Total: 4
Armijo: th(2.2605613425925934)=1.8581022582828517E-33; dx=5.874422239587826E-36 evalInputDelta=-1.4526693958844936E-33
Adding measurement 50c5f2a7 to history. Total: 4
New Minimum: 4.054328623983581E-34 > 1.6379178208149268E-34
END: th(1.1302806712962967)=1.6379178208149268E-34; dx=-2.0177886743866465E-36 evalInputDelta=2.416410803168654E-34
Fitness changed from 4.054328623983581E-34 to 1.6379178208149268E-34
Iteration 18 complete. Error: 1.6379178208149268E-34 Total: 0.5147; Orientation: 0.0838; Line Search: 0.3244
Non-optimal measurement 1.6379178208149268E-34 < 1.6379178208149268E-34. Total: 5
Rejected: LBFGS Orientation magnitude: 1.636e-18, gradient 9.891e-19, dot -0.997; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.6379178208149268E-34, 4.054328623983581E-34, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 1.636e-18, gradient 9.891e-19, dot -0.997; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.6379178208149268E-34, 4.054328623983581E-34, 0.18799758146127732, 74.38853294912732
LBFGS Accumulation History: 3 points
Removed measurement 50c5f2a7 to history. Total: 4
Removed measurement 28e720af to history. Total: 3
Adding measurement 148baa38 to history. Total: 3
th(0)=1.6379178208149268E-34;dx=-9.783534238028796E-37
Adding measurement bde77cc to history. Total: 4
New Minimum: 1.6379178208149268E-34 > 0.0
END: th(2.4351158877132666)=0.0; dx=0.0 evalInputDelta=1.6379178208149268E-34
Fitness changed from 1.6379178208149268E-34 to 0.0
Iteration 19 complete. Error: 0.0 Total: 0.3716; Orientation: 0.0485; Line Search: 0.2159
Non-optimal measurement 0.0 < 0.0. Total: 5
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 1.6379178208149268E-34, 0.18799758146127732, 74.38853294912732, 89.97905736583994
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 1.6379178208149268E-34, 0.18799758146127732, 74.38853294912732
LBFGS Accumulation History: 3 points
Removed measurement bde77cc to history. Total: 4
Removed measurement 148baa38 to history. Total: 3
Adding measurement 74996236 to history. Total: 3
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Non-optimal measurement 0.0 < 0.0. Total: 4
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.3713; Orientation: 0.0479; Line Search: 0.2163
Iteration 20 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 20
Final threshold in iteration 20: 0.0 (> 0.0) after 10.126s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.01 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -33.7857078918686], [19.0, -0.7258477377830105]; valueStats=DoubleSummaryStatistics{count=45, sum=0.379027, min=0.000000, average=0.008423, max=0.187998}
Plotting 19 points for GD
Plotting 10 points for CjGD
Plotting 19 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -33.7857078918686], [9.105, -0.7258477377830105]; valueStats=DoubleSummaryStatistics{count=45, sum=0.379027, min=0.000000, average=0.008423, max=0.187998}
Plotting 19 points for GD
Plotting 10 points for CjGD
Plotting 19 points for LBFGS
Returns
In this apply, attempt to train a network to emulate a randomized network given an example input/output. The target state is:
TrainingTester.java:279 executed in 0.00 seconds (0.000 gc):
RefList<double[]> temp_18_0037 = temp_18_0035.state();
assert temp_18_0037 != null;
String temp_18_0036 = temp_18_0037.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b).orElse("");
temp_18_0037.freeRef();
return temp_18_0036;
Returns
We simultaneously regress this target input:
TrainingTester.java:287 executed in 0.01 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(x -> {
RefStream<Tensor> temp_18_0006 = RefArrays.stream(RefUtil.addRef(x));
if (null != x)
RefUtil.freeRef(x);
return temp_18_0006;
}).map(x -> {
String temp_18_0007 = x.prettyPrint();
x.freeRef();
return temp_18_0007;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -1.036, -1.92, 0.644 ], [ 1.736, 0.124, -1.78 ], [ -1.316, 0.796, 1.412 ], [ 0.488, 0.82, -1.912 ], [ -0.376, 1.34, 1.724 ], [ -1.728, -1.668, -1.252 ], [ 0.112, -0.252, 1.772 ], [ -0.008, 1.688, -0.388 ], ... ],
[ [ 1.732, 0.668, 0.832 ], [ 0.216, 1.664, 0.828 ], [ -0.212, -0.476, -0.488 ], [ -0.828, -0.624, 0.924 ], [ 1.596, -0.448, -1.032 ], [ 1.848, -1.756, -0.476 ], [ -0.016, -0.06, 1.832 ], [ 1.432, -0.46, -0.464 ], ... ],
[ [ -1.916, 1.428, -0.912 ], [ 1.46, 0.884, 1.3 ], [ 1.588, 1.156, -1.308 ], [ -0.92, -1.14, -1.44 ], [ 0.184, -0.136, 0.824 ], [ 1.944, -1.744, 0.936 ], [ 1.636, -1.264, -1.164 ], [ 0.78, 1.216, 0.048 ], ... ],
[ [ 1.284, 0.588, 1.28 ], [ 1.876, -0.276, -1.896 ], [ 0.324, 0.068, -0.128 ], [ -1.276, 0.524, -0.272 ], [ 0.712, 0.928, -0.612 ], [ -0.892, -1.488, -1.136 ], [ 0.928, 1.936, 0.096 ], [ -1.956, -0.684, 0.292 ], ... ],
[ [ -1.452, -0.38, -1.628 ], [ -0.964, -0.544, 1.396 ], [ 0.488, 1.612, -0.792 ], [ 1.164, -1.712, 0.768 ], [ -1.796, -0.74, -0.888 ], [ 0.608, 1.384, 1.804 ], [ -1.216, 1.588, -0.716 ], [ -1.292, -1.404, -1.572 ], ... ],
[ [ 0.96, 1.312, -1.012 ], [ -0.984, -1.832, -0.128 ], [ 0.536, 1.288, -0.544 ], [ -1.66, -0.556, 1.752 ], [ 1.676, 0.268, -1.472 ], [ -0.064, 0.636, 0.18 ], [ -0.236, 0.464, -0.896 ], [ -0.268, 1.96, 1.312 ], ... ],
[ [ 0.864, 1.12, -1.24 ], [ -1.068, -1.2, 0.42 ], [ 1.976, 1.368, -1.48 ], [ 1.1, -0.26, -0.268 ], [ 1.716, -1.392, 0.348 ], [ -1.46, -0.648, 0.708 ], [ 1.7, 0.936, -1.684 ], [ -0.388, -0.908, -0.432 ], ... ],
[ [ -0.476, 1.812, 0.356 ], [ 0.936, -0.604, 1.284 ], [ 1.164, -0.972, 0.484 ], [ -1.628, -1.272, -0.736 ], [ -0.864, 1.68, 0.584 ], [ -0.992, -0.6, 1.6 ], [ -0.7, -0.936, -0.984 ], [ 1.66, 1.224, -0.852 ], ... ],
...
]
[
[ [ 1.052, 0.108, 0.088 ], [ -1.548, 0.152, -1.288 ], [ 0.848, -1.936, -1.388 ], [ 0.524, -0.592, -1.936 ], [ -0.252, 0.112, -0.856 ], [ -0.392, -0.572, 1.404 ], [ -1.032, 0.804, -1.392 ], [ -1.432, 0.444, -0.744 ], ... ],
[ [ -0.656, 1.46, 0.608 ], [ -1.284, -1.6, 1.368 ], [ 0.288, -1.324, 1.148 ], [ -0.996, 0.372, 0.492 ], [ 0.236, -1.624, 1.368 ], [ 0.168, 0.356, -1.16 ], [ -1.5, -1.172, 0.896 ], [ -1.552, 1.54, -0.236 ], ... ],
[ [ -0.988, 0.296, 0.5 ], [ 0.164, 0.892, -0.956 ], [ 1.816, -1.636, -0.024 ], [ 0.32, 0.052, -0.216 ], [ -1.852, 1.264, -1.612 ], [ -1.116, -1.032, -0.048 ], [ 0.7, 1.476, -0.66 ], [ -0.116, -0.788, -0.908 ], ... ],
[ [ 0.248, -0.408, -0.232 ], [ -0.88, -0.468, -0.332 ], [ 1.0, -1.676, 0.528 ], [ 0.728, 0.672, -1.476 ], [ -0.068, -0.616, -0.372 ], [ -1.892, 0.508, 0.896 ], [ 1.304, -1.18, 1.804 ], [ 1.572, 0.04, -0.156 ], ... ],
[ [ 0.956, 1.696, 1.508 ], [ -0.564, 1.864, 1.26 ], [ -1.568, 0.052, -0.764 ], [ -1.932, -0.856, 0.908 ], [ -0.2, -1.604, 0.008 ], [ 1.88, -1.376, 0.62 ], [ -2.0, -1.052, 0.26 ], [ -1.676, 0.048, -0.552 ], ... ],
[ [ 0.204, 1.208, 0.988 ], [ 1.092, -0.256, 1.184 ], [ 0.928, -0.756, -0.264 ], [ 0.548, 1.844, -1.568 ], [ -0.94, 0.932, -1.444 ], [ -1.964, 1.248, -1.052 ], [ -0.248, -1.556, 0.024 ], [ -1.824, 0.132, -0.488 ], ... ],
[ [ 1.4, 0.368, 0.072 ], [ -0.236, 0.676, 1.332 ], [ 0.316, -1.224, -1.272 ], [ -1.924, -0.06, 1.096 ], [ 0.848, 1.964, 0.004 ], [ -0.176, 0.532, 0.312 ], [ 1.744, 1.412, 0.872 ], [ -1.252, 1.568, 1.524 ], ... ],
[ [ 1.792, 0.876, 1.668 ], [ 0.648, 1.184, 0.512 ], [ -1.556, -0.652, -1.816 ], [ -1.98, -1.28, -1.92 ], [ -0.136, -1.548, -0.152 ], [ -0.356, 0.804, -1.856 ], [ 1.448, 1.196, -0.356 ], [ 1.176, 1.836, 1.988 ], ... ],
...
]
Which produces the following output:
TrainingTester.java:308 executed in 0.01 seconds (0.000 gc):
return RefStream.of(RefUtil.addRef(output_target)).map(x -> {
String temp_18_0008 = x.prettyPrint();
x.freeRef();
return temp_18_0008;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -1.1180479999999993, 5.9201120000000005, 4.150639999999999 ], [ -4.542768000000001, 6.254304, 6.510112 ], [ 3.4522399999999998, 4.333600000000001, -6.20256 ], [ -5.177296, -2.4045279999999987, 2.3786880000000012 ], [ 11.752048000000002, 0.051840000000000906, -2.103232 ], [ -0.3338559999999995, -1.7178879999999999, 0.8537279999999987 ], [ -2.118543999999999, 5.094704, 7.042896000000001 ], [ 5.737503999999999, -3.7653760000000003, -1.8621280000000002 ], ... ],
[ [ -2.0040160000000005, -3.1062400000000006, -7.595184 ], [ -0.2596000000000001, 3.9587840000000014, -2.9141920000000012 ], [ 5.297487999999999, 6.7402560000000005, -12.839567999999998 ], [ -0.22035199999999977, -7.810160000000002, 7.567856 ], [ 11.227360000000004, -7.171920000000003, 4.5051359999999985 ], [ 10.955744, -3.605904, 21.447904 ], [ 11.948784000000003, 4.283408, -7.2664 ], [ -2.753392, -6.670448, 4.38248 ], ... ],
[ [ -0.6650559999999999, 12.683919999999999, -7.376671999999998 ], [ 11.469968, 1.0524960000000005, 1.6887360000000011 ], [ -1.572576, -0.9096000000000006, 6.7992159999999995 ], [ -3.7191199999999993, 6.462288000000002, -7.239104 ], [ 11.739775999999999, 4.35264, 3.7576320000000005 ], [ 5.028351999999999, -0.06248000000000008, 6.608799999999998 ], [ -1.7027360000000002, 6.392512, -13.980512000000001 ], [ 0.6317279999999993, 2.89392, 3.305824000000002 ], ... ],
[ [ -2.5775680000000003, -12.593696, -2.1612479999999996 ], [ 0.9831679999999996, -7.306944000000002, -0.7785440000000007 ], [ 16.213583999999997, -4.310672, -2.2394559999999992 ], [ -2.4875840000000013, 11.254160000000002, 2.1033439999999994 ], [ -6.650624000000003, -1.5256960000000002, 1.664 ], [ -3.9452160000000003, 11.839199999999998, -3.3883040000000015 ], [ -3.079088, 11.643263999999999, -5.300912000000002 ], [ 0.1587040000000008, -2.4105919999999994, -16.063920000000003 ], ... ],
[ [ 3.763856000000001, 6.290656, -10.969008000000002 ], [ -1.292400000000001, 5.4041760000000005, 7.087728000000001 ], [ 1.32288, -3.0172800000000004, -11.058416000000001 ], [ -5.488144, -1.963552, 5.429056 ], [ 13.33576, -3.7736000000000005, -1.438208000000001 ], [ -11.062608000000003, 8.213888, -1.1851520000000004 ], [ -5.647615999999999, -1.9359199999999996, -8.876112 ], [ -13.319008000000002, 9.183936000000001, 6.171951999999999 ], ... ],
[ [ -1.1966400000000004, -2.5957280000000003, 3.4237119999999996 ], [ -5.590528000000001, 5.663679999999999, -4.437728 ], [ 3.30824, -1.0483520000000006, 2.7639679999999998 ], [ 13.837727999999998, 7.89408, -0.07412799999999928 ], [ -12.386703999999998, 2.3542399999999994, 6.777952 ], [ 2.6609919999999994, 1.0268480000000002, -6.283536 ], [ 8.964032000000001, -8.273551999999999, -0.7689600000000004 ], [ 0.45745600000000164, 4.097743999999998, 1.08864 ], ... ],
[ [ 1.9665759999999994, -3.212624, 1.9184640000000002 ], [ 3.2382559999999994, 7.455264, 0.8474399999999981 ], [ 4.3393440000000005, -0.17079999999999945, 10.370560000000001 ], [ 5.238448000000002, -3.061984, -4.688432 ], [ 0.7820160000000007, -16.42744, 1.504463999999999 ], [ 2.812608, 6.549648, -5.7080800000000025 ], [ -2.31688, -8.743072000000002, 9.975871999999999 ], [ 5.136431999999999, -1.0394240000000017, -11.327392000000001 ], ... ],
[ [ 5.532736000000001, -0.32862400000000025, -5.945008 ], [ 1.039968, -5.9030879999999994, -7.889456 ], [ 4.469119999999998, -3.8065600000000006, 0.6033119999999998 ], [ 5.993888, 13.032128000000002, -5.8619840000000005 ], [ -2.6610720000000008, 11.380463999999998, -3.347423999999999 ], [ -9.305855999999999, 7.3288480000000025, -10.681024000000003 ], [ 9.956496000000001, -8.279392, 10.319375999999998 ], [ 7.118688000000001, 1.4583840000000008, 3.0583840000000024 ], ... ],
...
]
[
[ [ -4.249584000000001, -3.9217440000000003, -0.9863199999999993 ], [ -0.39401600000000075, 0.7995520000000006, 3.6212159999999995 ], [ -0.5215519999999997, -3.7107999999999994, 14.371024 ], [ 1.0338880000000001, -1.2050400000000019, 4.246703999999999 ], [ 1.363184, 5.483888, 4.239488000000001 ], [ 3.213456, -5.600112, -2.4782879999999996 ], [ -15.639968000000001, 3.5793440000000007, 0.5281439999999998 ], [ -4.127648000000001, -0.9506239999999989, -2.6916480000000016 ], ... ],
[ [ 1.6960479999999998, 1.14144, -4.445328 ], [ 8.136351999999999, -5.721984, 0.12214400000000017 ], [ 9.19544, 1.4129119999999988, 10.100511999999998 ], [ 1.3193280000000003, 9.00704, -6.479424000000002 ], [ -3.4016799999999985, -7.095856, -3.5096160000000003 ], [ -5.326944000000001, -0.1625600000000011, -3.3957759999999997 ], [ -1.6762240000000017, -4.654432000000001, -3.3927360000000006 ], [ -3.922495999999997, 3.1311839999999997, 7.043455999999996 ], ... ],
[ [ -1.4522560000000002, -0.4286880000000002, 0.5845920000000001 ], [ 1.6720159999999997, -3.30928, 9.734528000000001 ], [ 4.192847999999998, -1.3631040000000008, 9.803424000000001 ], [ -1.4941120000000003, 3.404176, -1.0096640000000003 ], [ -7.904000000000002, 8.898784000000001, -5.322063999999999 ], [ -9.284320000000003, -4.504079999999997, 4.635648000000001 ], [ -1.4486559999999997, 4.525792000000001, 7.030272000000001 ], [ 6.324832, 0.19507199999999897, 3.415696 ], ... ],
[ [ -9.671408, 4.267744, -1.473456 ], [ -0.6609440000000004, 7.341151999999999, 6.538911999999998 ], [ 5.54392, -3.7695519999999996, 4.556895999999998 ], [ -0.30374399999999946, -2.0800319999999983, -6.546415999999998 ], [ 4.721664, -13.024128000000001, 4.135712 ], [ 6.358624, -0.7585600000000009, -2.6226080000000005 ], [ 5.738592000000001, -12.5096, 7.407151999999999 ], [ -3.637215999999999, -11.288640000000003, -0.5370239999999996 ], ... ],
[ [ 1.8094560000000002, -1.540304, -4.001488 ], [ -0.0504319999999999, 5.71968, -0.3749279999999998 ], [ -0.45979199999999937, 8.229856, -1.0702400000000005 ], [ -3.1078400000000026, 8.782928000000002, -4.459119999999999 ], [ -5.492367999999999, 1.8366080000000005, 1.8120159999999992 ], [ -9.635344000000002, -7.018112, -0.31142399999999976 ], [ -2.380688, 2.1032159999999993, -2.5121440000000006 ], [ -6.748768000000001, 1.4007840000000003, -4.516768000000002 ], ... ],
[ [ 0.49761599999999984, -1.3905120000000004, -9.4344 ], [ 8.026767999999999, -6.00088, 5.2456320000000005 ], [ 5.856528, -14.237295999999999, 12.998400000000002 ], [ -13.114175999999999, 3.697344000000002, 2.585664 ], [ -9.991439999999997, 7.507600000000002, -2.1005439999999993 ], [ -5.693455999999999, 16.452832, -0.4898719999999992 ], [ -15.524656000000002, 8.652592000000002, 1.7954880000000002 ], [ -17.455392, 8.215312, 4.765023999999999 ], ... ],
[ [ -3.63104, 4.232704, 0.23355199999999934 ], [ 9.91544, 9.869920000000002, 1.5039679999999995 ], [ 0.5158079999999997, -3.72864, -3.648592 ], [ 3.6552320000000007, -8.894704000000003, -15.338592 ], [ -0.24652799999999883, -10.729488, 0.06793599999999983 ], [ 5.414528000000001, -10.392688, -5.825488000000001 ], [ 0.14526400000000037, -5.082319999999999, -2.0150720000000018 ], [ 0.3548480000000003, 8.013296, -9.30376 ], ... ],
[ [ 1.7201120000000005, -10.114304, -2.2403839999999997 ], [ -4.3257759999999985, -5.844095999999999, -10.873008 ], [ -10.4456, 2.6564320000000006, 12.122160000000001 ], [ -14.334688, 10.515584, -3.283728000000001 ], [ -6.674256000000001, 2.4589919999999994, 6.7984 ], [ -6.1567359999999995, 9.901952000000003, -2.183967999999999 ], [ 0.36811200000000155, -2.887280000000001, -5.877999999999999 ], [ -0.07494400000000084, -3.6549599999999995, -9.997760000000001 ], ... ],
...
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 30.19 seconds (0.295 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13586643802407
Reset training subject: 13586793750524
Constructing line search parameters: GD
th(0)=94.57237528495472;dx=-84.05354178633752
New Minimum: 94.57237528495472 > 86.74823548046467
WOLF (strong): th(2.154434690031884)=86.74823548046467; dx=76.76554836696447 evalInputDelta=7.824139804490045
New Minimum: 86.74823548046467 > 47.35102751490329
END: th(1.077217345015942)=47.35102751490329; dx=-3.6254676375286494 evalInputDelta=47.22134777005142
Fitness changed from 94.57237528495472 to 47.35102751490329
Iteration 1 complete. Error: 47.35102751490329 Total: 1.1693; Orientation: 0.0739; Line Search: 0.6454
th(0)=47.35102751490329;dx=-0.1588224204384863
Armijo: th(2.3207944168063896)=47.36177727876125; dx=0.1680861893480153 evalInputDelta=-0.010749763857958783
New Minimum: 47.35102751490329 > 47.26156643728305
WOLF (strong): th(1.1603972084031948)=47.26156643728305; dx=0.004631957022192299 evalInputDelta=0.08946107762024269
END: th(0.3867990694677316)=47.30013248762287; dx=-0.1043376118247544 evalInputDelta=0.05089502728042561
Fitness changed from 47.35102751490329 to 47.26156643728305
Iteration 2 complete. Error: 47.26156643728305 Total: 1.0379; Orientation: 0.0715; Line Search: 0.8165
th(0)=47.26156643728305;dx=-1.4040099591331256E-4
New Minimum: 47.26156643728305 > 47.261492699002034
END: th(0.8333333333333335)=47.261492699002034; dx=-3.657087856207661E-5 evalInputDelta=7.373828101719937E-5
Fitness changed from 47.26156643728305 to 47.261492699002034
Iteration 3 complete. Error: 47.261492699002034 Total: 0.6357; Orientation: 0.0738; Line Search: 0.4128
th(0)=47.261492699002034;dx=-9.643135058399481E-6
New Minimum: 47.261492699002034 > 47.26148889544319
WOLF (strong): th(1.7953622416932369)=47.26148889544319; dx=5.40604156978444E-6 evalInputDelta=3.803558840331789E-6
New Minimum: 47.26148889544319 > 47.26148741988218
END: th(0.8976811208466184)=47.26148741988218; dx=-2.118546741840777E-6 evalInputDelta=5.279119854151304E-6
Fitness changed from 47.261492699002034 to 47.26148741988218
Iteration 4 complete. Error: 47.26148741988218 Total: 0.8459; Orientation: 0.0711; Line Search: 0.6264
th(0)=47.26148741988218;dx=-5.964392640497723E-7
New Minimum: 47.26148741988218 > 47.26148690810365
WOLF (strong): th(1.9339953473386584)=47.26148690810365; dx=6.719442919679247E-8 evalInputDelta=5.117785306651967E-7
END: th(0.9669976736693292)=47.26148700355985; dx=-2.646224173228984E-7 evalInputDelta=4.1632232949950776E-7
Fitness changed from 47.26148741988218 to 47.26148690810365
Iteration 5 complete. Error: 47.26148690810365 Total: 0.8377; Orientation: 0.0721; Line Search: 0.6153
th(0)=47.26148690810365;dx=-4.0948652050352633E-7
New Minimum: 47.26148690810365 > 47.26148643919406
END: th(2.083333333333334)=47.26148643919406; dx=-4.066669630696155E-8 evalInputDelta=4.689095902676854E-7
Fitness changed from 47.26148690810365 to 47.26148643919406
Iteration 6 complete. Error: 47.26148643919406 Total: 0.6361; Orientation: 0.0726; Line Search: 0.4128
th(0)=47.26148643919406;dx=-3.5464001680278E-7
New Minimum: 47.26148643919406 > 47.26148613949037
WOLF (strong): th(4.488405604233092)=47.26148613949037; dx=2.2109429881918854E-7 evalInputDelta=2.997036858687352E-7
New Minimum: 47.26148613949037 > 47.26148596632608
END: th(2.244202802116546)=47.26148596632608; dx=-6.677285877126278E-8 evalInputDelta=4.728679812160408E-7
Fitness changed from 47.26148643919406 to 47.26148596632608
Iteration 7 complete. Error: 47.26148596632608 Total: 0.8469; Orientation: 0.0709; Line Search: 0.6265
th(0)=47.26148596632608;dx=-3.5307098263075895E-7
New Minimum: 47.26148596632608 > 47.26148574184351
WOLF (strong): th(4.834988368346646)=47.26148574184351; dx=2.6021344008342206E-7 evalInputDelta=2.2448256942197986E-7
New Minimum: 47.26148574184351 > 47.261485483431905
END: th(2.417494184173323)=47.261485483431905; dx=-4.6428770995327715E-8 evalInputDelta=4.828941726486846E-7
Fitness changed from 47.26148596632608 to 47.261485483431905
Iteration 8 complete. Error: 47.261485483431905 Total: 0.8453; Orientation: 0.0704; Line Search: 0.6263
th(0)=47.261485483431905;dx=-3.9870447811054297E-7
Armijo: th(5.208333333333335)=47.26148567688161; dx=4.7298916266173065E-7 evalInputDelta=-1.9344970780821313E-7
New Minimum: 47.261485483431905 > 47.26148501264788
WOLF (strong): th(2.6041666666666674)=47.26148501264788; dx=3.714234268730744E-8 evalInputDelta=4.7078402332090263E-7
END: th(0.8680555555555558)=47.26148520039081; dx=-2.534222044196355E-7 evalInputDelta=2.830410963383656E-7
Fitness changed from 47.261485483431905 to 47.26148501264788
Iteration 9 complete. Error: 47.26148501264788 Total: 1.4778; Orientation: 0.1514; Line Search: 1.1765
th(0)=47.26148501264788;dx=-5.359702189708683E-7
New Minimum: 47.26148501264788 > 47.26148451629145
WOLF (strong): th(1.8701690017637886)=47.26148451629145; dx=5.155704604820192E-9 evalInputDelta=4.963564279591992E-7
END: th(0.9350845008818943)=47.26148463797005; dx=-2.6540725708301505E-7 evalInputDelta=3.7467783187139503E-7
Fitness changed from 47.26148501264788 to 47.26148451629145
Iteration 10 complete. Error: 47.26148451629145 Total: 0.9035; Orientation: 0.0763; Line Search: 0.6713
th(0)=47.26148451629145;dx=-3.5338409875637686E-7
New Minimum: 47.26148451629145 > 47.26148406212049
END: th(2.0145784868111027)=47.26148406212049; dx=-9.750025796183189E-8 evalInputDelta=4.541709657246429E-7
Fitness changed from 47.26148451629145 to 47.26148406212049
Iteration 11 complete. Error: 47.26148406212049 Total: 0.6373; Orientation: 0.0726; Line Search: 0.4148
th(0)=47.26148406212049;dx=-2.999859355108606E-7
New Minimum: 47.26148406212049 > 47.26148350898385
WOLF (strong): th(4.3402777777777795)=47.26148350898385; dx=4.5100570710436884E-8 evalInputDelta=5.53136636938234E-7
END: th(2.1701388888888897)=47.261483598330756; dx=-1.274426822412105E-7 evalInputDelta=4.6378973195260187E-7
Fitness changed from 47.26148406212049 to 47.26148350898385
Iteration 12 complete. Error: 47.26148350898385 Total: 0.8394; Orientation: 0.0727; Line Search: 0.6168
th(0)=47.26148350898385;dx=-9.418491885318841E-7
Armijo: th(4.675422504409472)=47.261486211957134; dx=2.0980968972477715E-6 evalInputDelta=-2.7029732834193965E-6
New Minimum: 47.26148350898385 > 47.26148308384144
WOLF (strong): th(2.337711252204736)=47.26148308384144; dx=5.781238558060228E-7 evalInputDelta=4.251424101653356E-7
New Minimum: 47.26148308384144 > 47.261482972463256
END: th(0.7792370840682453)=47.261482972463256; dx=-4.3519150676381596E-7 evalInputDelta=5.365205950624841E-7
Fitness changed from 47.26148350898385 to 47.261482972463256
Iteration 13 complete. Error: 47.261482972463256 Total: 1.0507; Orientation: 0.0721; Line Search: 0.8290
th(0)=47.261482972463256;dx=-2.7963280978032534E-7
New Minimum: 47.261482972463256 > 47.261482589389146
END: th(1.678815405675919)=47.261482589389146; dx=-1.767295922269061E-7 evalInputDelta=3.8307410932247876E-7
Fitness changed from 47.261482972463256 to 47.261482589389146
Iteration 14 complete. Error: 47.261482589389146 Total: 0.6382; Orientation: 0.0721; Line Search: 0.4161
th(0)=47.261482589389146;dx=-2.2746479839246132E-7
New Minimum: 47.261482589389146 > 47.26148186397921
END: th(3.6168981481481497)=47.26148186397921; dx=-1.7365787568829225E-7 evalInputDelta=7.254099330111785E-7
Fitness changed from 47.261482589389146 to 47.26148186397921
Iteration 15 complete. Error: 47.26148186397921 Total: 0.6353; Orientation: 0.0735; Line Search: 0.4114
th(0)=47.26148186397921;dx=-2.9306775551087815E-7
New Minimum: 47.26148186397921 > 47.26148180096414
WOLF (strong): th(7.7923708406824534)=47.26148180096414; dx=2.768942247956307E-7 evalInputDelta=6.301507227135517E-8
New Minimum: 47.26148180096414 > 47.261481277302295
END: th(3.8961854203412267)=47.261481277302295; dx=-8.086764909191377E-9 evalInputDelta=5.866769186013698E-7
Fitness changed from 47.26148186397921 to 47.261481277302295
Iteration 16 complete. Error: 47.261481277302295 Total: 0.9918; Orientation: 0.0707; Line Search: 0.7713
th(0)=47.261481277302295;dx=-7.1079698616218E-7
Armijo: th(8.394077028379595)=47.26149097447001; dx=3.0212755916251826E-6 evalInputDelta=-9.697167712374721E-6
Armijo: th(4.19703851418
...skipping 1391 bytes...
995064190316331)=47.26148252554832; dx=1.3633702098139798E-6 evalInputDelta=-2.976046459934878E-6
New Minimum: 47.26147954950186 > 47.26147939732117
WOLF (strong): th(3.4975320951581654)=47.26147939732117; dx=4.254494861320081E-7 evalInputDelta=1.5218068938338547E-7
New Minimum: 47.26147939732117 > 47.2614791342852
END: th(1.1658440317193886)=47.2614791342852; dx=-1.998309979894848E-7 evalInputDelta=4.1521665394839147E-7
Fitness changed from 47.26147954950186 to 47.2614791342852
Iteration 20 complete. Error: 47.2614791342852 Total: 1.0514; Orientation: 0.0704; Line Search: 0.8315
th(0)=47.2614791342852;dx=-2.1155612831608126E-7
New Minimum: 47.2614791342852 > 47.261478604085625
WOLFE (weak): th(2.5117348251028826)=47.261478604085625; dx=-2.106218595627002E-7 evalInputDelta=5.301995784634528E-7
New Minimum: 47.261478604085625 > 47.26147807623268
WOLFE (weak): th(5.023469650205765)=47.26147807623268; dx=-2.096875908104185E-7 evalInputDelta=1.0580525255932116E-6
New Minimum: 47.26147807623268 > 47.261475988287266
WOLFE (weak): th(15.070408950617296)=47.261475988287266; dx=-2.059505158121625E-7 evalInputDelta=3.1459979368264612E-6
New Minimum: 47.261475988287266 > 47.2614670571667
END: th(60.28163580246918)=47.2614670571667; dx=-1.891336785272192E-7 evalInputDelta=1.2077118505260387E-5
Fitness changed from 47.2614791342852 to 47.2614670571667
Iteration 21 complete. Error: 47.2614670571667 Total: 1.2817; Orientation: 0.0716; Line Search: 1.0606
th(0)=47.2614670571667;dx=-1.4112444142208932E-6
Armijo: th(129.8728473447076)=47.27029266148864; dx=1.3732270391600477E-4 evalInputDelta=-0.008825604321941682
Armijo: th(64.9364236723538)=47.26362763771677; dx=6.795573212787025E-5 evalInputDelta=-0.002160580550075508
Armijo: th(21.645474557451266)=47.26168675697284; dx=2.171108164475341E-5 evalInputDelta=-2.1969980614500173E-4
Armijo: th(5.411368639362816)=47.26147506083185; dx=4.369337152473663E-6 evalInputDelta=-8.003665151079531E-6
New Minimum: 47.2614670571667 > 47.261466155431094
END: th(1.0822737278725634)=47.261466155431094; dx=-2.551280980955313E-7 evalInputDelta=9.017356035201374E-7
Fitness changed from 47.2614670571667 to 47.261466155431094
Iteration 22 complete. Error: 47.261466155431094 Total: 1.4335; Orientation: 0.0717; Line Search: 1.2115
th(0)=47.261466155431094;dx=-2.1417532606536954E-7
New Minimum: 47.261466155431094 > 47.26146565971847
WOLFE (weak): th(2.3316880634387775)=47.26146565971847; dx=-2.1102102786906066E-7 evalInputDelta=4.957126265026091E-7
New Minimum: 47.26146565971847 > 47.26146517136067
WOLFE (weak): th(4.663376126877555)=47.26146517136067; dx=-2.0786672967414565E-7 evalInputDelta=9.840704251473653E-7
New Minimum: 47.26146517136067 > 47.26146329147791
WOLFE (weak): th(13.990128380632665)=47.26146329147791; dx=-1.952495369081082E-7 evalInputDelta=2.8639531848284605E-6
New Minimum: 47.26146329147791 > 47.26145628826362
END: th(55.96051352253066)=47.26145628826362; dx=-1.3847216970351108E-7 evalInputDelta=9.8671674706452E-6
Fitness changed from 47.261466155431094 to 47.26145628826362
Iteration 23 complete. Error: 47.26145628826362 Total: 1.2737; Orientation: 0.0715; Line Search: 1.0529
th(0)=47.26145628826362;dx=-3.868010808189019E-6
Armijo: th(120.5632716049384)=47.284626816269444; dx=3.882392580044371E-4 evalInputDelta=-0.023170528005820756
Armijo: th(60.2816358024692)=47.26713233537459; dx=1.9218562943820213E-4 evalInputDelta=-0.00567604711096692
Armijo: th(20.093878600823064)=47.2620351446141; dx=6.148320394368122E-5 evalInputDelta=-5.788563504793842E-4
Armijo: th(5.023469650205766)=47.261477893659574; dx=1.246979300705746E-5 evalInputDelta=-2.1605395950530237E-5
New Minimum: 47.26145628826362 > 47.26145404354588
END: th(1.0046939300411533)=47.26145404354588; dx=-6.004500383146888E-7 evalInputDelta=2.2447177414619546E-6
Fitness changed from 47.26145628826362 to 47.26145404354588
Iteration 24 complete. Error: 47.26145404354588 Total: 1.4395; Orientation: 0.0709; Line Search: 1.2187
th(0)=47.26145404354588;dx=-2.5499865555055044E-7
New Minimum: 47.26145404354588 > 47.26145357688291
END: th(2.164547455745127)=47.26145357688291; dx=-1.7618891074260205E-7 evalInputDelta=4.6666297492947706E-7
Fitness changed from 47.26145404354588 to 47.26145357688291
Iteration 25 complete. Error: 47.26145357688291 Total: 0.6377; Orientation: 0.0719; Line Search: 0.4146
th(0)=47.26145357688291;dx=-2.492029768159375E-7
New Minimum: 47.26145357688291 > 47.261452754100354
END: th(4.663376126877556)=47.261452754100354; dx=-1.0366693350920785E-7 evalInputDelta=8.227825532003408E-7
Fitness changed from 47.26145357688291 to 47.261452754100354
Iteration 26 complete. Error: 47.261452754100354 Total: 0.6380; Orientation: 0.0716; Line Search: 0.4148
th(0)=47.261452754100354;dx=-5.622701978934517E-7
Armijo: th(10.046939300411534)=47.26146273498415; dx=2.54912081601237E-6 evalInputDelta=-9.98088379589035E-6
Armijo: th(5.023469650205767)=47.26145383704767; dx=9.934253128034986E-7 evalInputDelta=-1.0829473140461232E-6
New Minimum: 47.261452754100354 > 47.26145224675066
END: th(1.6744898834019224)=47.26145224675066; dx=-4.3705026827061626E-8 evalInputDelta=5.07349696476922E-7
Fitness changed from 47.261452754100354 to 47.26145224675066
Iteration 27 complete. Error: 47.26145224675066 Total: 1.0357; Orientation: 0.0716; Line Search: 0.8144
th(0)=47.26145224675066;dx=-2.9786616845223664E-7
New Minimum: 47.26145224675066 > 47.26145165621158
END: th(3.607579092908546)=47.26145165621158; dx=-2.9521848365304924E-8 evalInputDelta=5.905390807470212E-7
Fitness changed from 47.26145224675066 to 47.26145165621158
Iteration 28 complete. Error: 47.26145165621158 Total: 0.6331; Orientation: 0.0714; Line Search: 0.4104
th(0)=47.26145165621158;dx=-6.229124927563989E-7
Armijo: th(7.772293544795928)=47.26145779750692; dx=2.203217020245891E-6 evalInputDelta=-6.141295344264108E-6
Armijo: th(3.886146772397964)=47.26145198117074; dx=7.901522664941558E-7 evalInputDelta=-3.2495916002517333E-7
New Minimum: 47.26145165621158 > 47.26145115437828
END: th(1.295382257465988)=47.26145115437828; dx=-1.5189090572747715E-7 evalInputDelta=5.018332984718654E-7
Fitness changed from 47.26145165621158 to 47.26145115437828
Iteration 29 complete. Error: 47.26145115437828 Total: 1.0357; Orientation: 0.0718; Line Search: 0.8136
th(0)=47.26145115437828;dx=-2.2396856063168332E-7
New Minimum: 47.26145115437828 > 47.26145056246612
END: th(2.7908164723365374)=47.26145056246612; dx=-2.0021709845994052E-7 evalInputDelta=5.919121548458861E-7
Fitness changed from 47.26145115437828 to 47.26145056246612
Iteration 30 complete. Error: 47.26145056246612 Total: 0.6338; Orientation: 0.0708; Line Search: 0.4132
th(0)=47.26145056246612;dx=-2.3557516906558094E-7
New Minimum: 47.26145056246612 > 47.2614494855829
END: th(6.012631821514243)=47.2614494855829; dx=-1.2263176915737813E-7 evalInputDelta=1.0768832225949154E-6
Fitness changed from 47.26145056246612 to 47.2614494855829
Iteration 31 complete. Error: 47.2614494855829 Total: 0.6321; Orientation: 0.0716; Line Search: 0.4113
th(0)=47.2614494855829;dx=-6.14818602179789E-7
Armijo: th(12.95382257465988)=47.261471421838216; dx=4.001657524053656E-6 evalInputDelta=-2.1936255315324615E-5
Armijo: th(6.47691128732994)=47.26145297858398; dx=1.6934194684716632E-6 evalInputDelta=-3.4930010812672663E-6
New Minimum: 47.2614494855829 > 47.26144898877735
WOLF (strong): th(2.15897042910998)=47.26144898877735; dx=1.545940897173022E-7 evalInputDelta=4.968055478116185E-7
END: th(0.539742607277495)=47.26144920564971; dx=-4.224654290479218E-7 evalInputDelta=2.799331895175783E-7
Fitness changed from 47.2614494855829 to 47.26144898877735
Iteration 32 complete. Error: 47.26144898877735 Total: 1.2340; Orientation: 0.0726; Line Search: 1.0126
th(0)=47.26144898877735;dx=-5.553802719357716E-7
New Minimum: 47.26144898877735 > 47.26144854810845
END: th(1.1628401968068907)=47.26144854810845; dx=-2.0253797229730948E-7 evalInputDelta=4.40668905810071E-7
Fitness changed from 47.26144898877735 to 47.26144854810845
Iteration 33 complete. Error: 47.26144854810845 Total: 0.6376; Orientation: 0.0716; Line Search: 0.4170
Final threshold in iteration 33: 47.26144854810845 (> 0.0) after 30.193s (< 30.000s)
Returns
47.26144854810845
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[-0.008660365166043371, 0.00340237799505878, -0.0071790075268749415, -0.0054758970347345386, 0.008852185922984507, -0.014756446327402426, -0.007827511217628197, -0.00945496268063516, -0.007626363592957261, -3.5578488119990964E-4, -0.0036998637071272403, -0.00619834843453379, 0.012014931202856634, -0.004244676145917312, 0.008756087685795978, 0.002849295542386343, -0.007277401540587329, 0.006035130609716288, 0.011834153840817862, 1.6372538771335674E-4, -5.050538077067078E-4, -0.0010912851987696214, 0.0011988811475068624, 0.004134063212435866, 0.010206883762724252, 4.516581625730228E-4, -0.013167526422146145, -0.006099819472161601, -0.010332652607502359, -0.003389340553924223, -0.002227080981932565, -0.020163769201830408, 0.008935501939394007, -0.011030272020135672, -7.986311276635294E-4, -6.53873420927432E-4, -0.0102702345919232, 0.007630531509007502, -0.01016980323629445, -0.01251096988193125, 0.007053241072507577, -0.004601444338199008, -0.010719223737195622, 0.015412459697256351, 0.0034818875088792507, -0.0018771381834046983, 0.009245278730056658, -0.009575921549666633, -0.0019243737298732615, 0.007091466209396547, -0.012945491783433197, -0.012975212754355257, 0.00936261312971474, -0.015940954741597405, 0.002118489628361786, 0.0063439503066510415, 0.00824014227205092, 0.002817369772162496, -0.0015216903942953024, -6.235651014985217E-4, 0.0028632738814464397, 0.008104541183030141, 0.0054805193090264, -0.008586119624593009, 0.005662126574304642, 0.0035837880512261597, -7.756534368672092E-4, -0.00112683490205511, 0.004278536512427871, 0.0010824711638413769, 0.01798829427160801, -0.00899753004569339, -0.006008598713366944, -0.0050144688263574514, -0.003440028023247002, -0.001170620742611845, 0.004207491166584983, -0.005074946063263197, 0.0035556787528524835, -0.011909396916771514, 0.0024200849140525744]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.5440499089323233, 0.5240261251997066, 1.0000250552660055 ], [ -1.4799992282034828, 3.197795262721623E-5, 1.3200143199843408 ], [ -1.8279051930491867, 1.1399494968564716, 0.1240562279869762 ], [ -1.7120562598615006, 0.29600370993284403, 1.7398720578198736 ], [ 0.5639708426866882, 1.8479770085367118, -1.3599717238529634 ], [ 0.7838968702146709, -0.8358489054194784, -0.1720612985791122 ], [ -0.98405301786194, -0.7800402109227199, 0.020072125514270556 ], [ 1.3559656536050733, 0.1640372103360191, -0.05605374859336213 ], ... ],
[ [ -1.6798801123221985, -1.520064774356997, -0.20407441647677563 ], [ -1.3839639534411508, 0.35585197753835357, 1.9719781395560236 ], [ -0.019961825458307113, -0.65199047753456, -1.9799469366182219 ], [ 0.6799256448839954, 0.8918431971390142, -0.7479828862883189 ], [ 0.02408713612593182, 1.3400266557526985, 0.471885162178966 ], [ 0.7398616822140218, 0.9720023354097257, 0.807989727712613 ], [ -0.7799302913170925, -0.10003645510268487, -1.6479436060602664 ], [ 0.7439424707360945, 0.5679864824333881, 1.7798935631657704 ], ... ],
[ [ 0.5159628084160324, 1.6599856597952252, 1.164069717749226 ], [ -1.851954447983542, -0.8039085578437186, 1.6158492197444239 ], [ -0.18797721932377512, 1.8160860722407417, 0.1840523563007309 ], [ 1.5199755596198632, -0.22788738140235718, 0.8760243658260369 ], [ 1.5519726097667368, -0.7000233881179194, 1.2879854719890427 ], [ -0.8760811324254748, -0.8040405553458897, -0.7798926532818127 ], [ 1.7399471324857902, -0.3319401228509462, 0.9919996347140804 ], [ -1.8120723773790743, -1.340026385968218, -0.9638870402774515 ], ... ],
[ [ -0.8999279656619249, -0.8760965865938732, 1.5639574649638888 ], [ 0.6759883863672475, -1.4719372018008274, 1.647827118099378 ], [ -1.0798206025075845, 1.511909465756356, 1.6318909679622797 ], [ 1.511808791547587, -0.27594512410140604, -0.8719378608418784 ], [ -1.616030282748997, 0.6600278943850991, 1.2160715728749587 ], [ -0.24002204326421192, 1.5920245417266745, -0.21991861625247888 ], [ 1.3919430049245138, -0.7361574031060968, -0.9517879471959745 ], [ 1.1680388526519554, -0.08005750466886379, -0.6000946673531629 ], ... ],
[ [ 0.9880906271152077, 0.2720887392093695, 1.352039719093372 ], [ 1.6798224246866016, -1.7920242765458374, 0.13217825459846477 ], [ -0.019971520491976983, -0.12795731635005966, 1.475831732310262 ], [ 1.9758977544260743, -0.10806228815994566, 0.36414661907672147 ], [ 0.7360500416947713, -1.6280336653616858, -0.4721138427520949 ], [ 1.9159210265734787, -1.1601396884818505, -0.9559589531761834 ], [ -1.6559418440966227, -0.8039002603196332, -0.7319656589404074 ], [ 0.5758628346305387, 1.8920473809357454, -0.2199069219236332 ], ... ],
[ [ -1.5519811247582678, 1.0000042420918416, 1.4359806748314343 ], [ -1.3440284298815681, 0.5039946046194059, -1.1839029727449388 ], [ 1.1198811265494009, 0.07596260036937523, -0.959960060801275 ], [ 0.37195934377732087, -0.9439033734106609, 1.0520372349454639 ], [ -1.4361187098882797, 1.7119860724842815, 0.18000362194474775 ], [ -1.071933033366532, 1.363980071878558, 1.927962493929201 ], [ -0.19588723041668207, -0.14396399561089956, 1.7317094811010934 ], [ -1.2519879962663962, -0.9520892559173992, -1.5319364251530585 ], ... ],
[ [ -1.9759179973711574, -1.5399810131732985, 0.283940695928212 ], [ -0.43587665145265575, -0.955927902897131, -0.6840638030074362 ], [ -1.4880078169875182, -0.9959604073600339, 0.5039588960995917 ], [ 1.639967757059775, -0.10402075125430614, 1.483956721450468 ], [ -1.835874095185776, 0.479889340858602, 1.4798431095552804 ], [ -0.6438918108776316, 0.3879556013865487, -1.684068338589238 ], [ -1.4840833859838516, -0.8399830325972268, 1.1759205629424334 ], [ -0.028005989843164963, -1.2398550877927255, -0.6279486874503212 ], ... ],
[ [ 0.6080044356005927, 0.05988724890134736, -0.7599548002631363 ], [ -1.235965876237339, 1.9118564686840622, -0.2679012978522894 ], [ -1.8079095931839568, 0.4599777507070653, 1.8999888950434043 ], [ 1.6839663810139238, 1.0080045125702743, -1.9317569928689375 ], [ 1.2077829100989368, -1.0120138334913558, -1.1798389979950523 ], [ 1.8159045067238657, 1.3959805909762941, 1.872160142855619 ], [ -1.7559859343845263, 0.29194107788421003, -1.99597043230988 ], [ 1.7919447596961466, -1.8278631532557312, -1.7839493049283213 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.02 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 0.02532490503535087, 0.05155109251510396, -0.03436983349297084 ], [ -0.03703020366871909, 0.12778194789122327, 0.007756136633258327 ], [ -0.001563424899186418, 0.08132619604904896, -0.0391267308625589 ], [ -0.019627186125570913, 0.07171015491469512, 0.014673469624934299 ], [ 0.023914925291330442, -0.04292458668289209, -0.04973569856712273 ], [ -0.0070934915323443495, -0.030459611838022604, 0.054964610647536126 ], [ 0.021726063055126272, -0.050530411493534384, -0.002502843564011084 ], [ 0.0082335076059298, -0.02305002983609622, -0.05063638177180912 ], ... ],
[ [ 0.04758254333148476, 0.03119331750506323, -0.04107777955261778 ], [ 0.051063816303734834, -0.0994088003599477, -0.055077803656792795 ], [ 0.10007718414967155, -0.08060042250535382, -0.04398138064756989 ], [ 0.03855363118997182, -0.09733062148953527, 0.008430431746559188 ], [ -0.04857865765001574, 0.014000491302126012, 0.06808428537436603 ], [ -9.606986659855917E-5, 0.04479577255671007, -4.7518704759508316E-4 ], [ -0.04637324053507002, 0.07991656282677785, 0.03906790950880868 ], [ -0.009812813287048347, -0.02052720926469308, 0.023654961701183277 ], ... ],
[ [ 0.020089134115421483, 0.03216977749099453, -0.04301142234133501 ], [ 0.03384552573835858, 0.08264805797656188, 0.017611534754431127 ], [ -0.009480151470053665, -0.029734030652051203, -0.02222453168271751 ], [ 0.06737877058712999, -0.005828322161951993, 0.03498046159615831 ], [ 0.010275860358981832, -0.06939677148321871, -0.01896613784356564 ], [ 0.02727873793877618, -0.019730282437042706, -0.010767478763245629 ], [ -0.014384570946708381, -0.07909330618476319, -0.004267487705960489 ], [ -0.021042581927877956, 0.040563985288428514, -0.0014383908705012122 ], ... ],
[ [ 0.01887559302920287, 6.607241744019015E-4, -0.007057801888214264 ], [ 0.022172999484550007, -0.05343066987562218, -0.04995021877184587 ], [ -0.03815175493641973, -0.054420426871644244, -1.9435785791429251E-4 ], [ -0.02161707922211491, 0.021944716630647274, -0.008054490215062556 ], [ -0.13423977293350986, 0.04441174101296237, 0.05817150080980504 ], [ -0.06474943006434841, 0.025798190799091077, 0.0339700273647669 ], [ -0.017634327902353142, 0.004199933573925297, -0.005486133400059717 ], [ -0.02348093019135937, -0.07303226149496804, -0.006215283559910655 ], ... ],
[ [ 0.053488897427662746, -0.07215663897526325, -0.019662143815073686 ], [ -0.0016664441404652698, -0.0989665152425815, -0.02243038123188712 ], [ 0.003281169350008986, -0.07052899706246203, 0.02982949196449395 ], [ -0.01585349390567392, -0.05832556332020013, -0.0703778812422526 ], [ 0.09567739764266088, -0.13882525666676057, 0.0022757217065432498 ], [ 0.09490413288191107, -0.11406687854350596, -0.043921557794975266 ], [ -0.02901650377232482, 0.03195922126771647, 0.03542373306563039 ], [ 0.022764930160654357, -0.010586913425030152, 0.01770271347416706 ], ... ],
[ [ -0.04249353172742386, 0.0935386945438665, 0.01488185422879766 ], [ -0.0426476884737155, 0.08724361688561016, -0.015323225024970064 ], [ -0.019181240030423003, -0.04262632988001955, 0.017176899162377267 ], [ -0.02246551213808332, 0.04105660487702832, 0.011920798880438073 ], [ -0.07625336630426534, 0.06590067859670673, -0.003646579859670934 ], [ 0.021954086480330072, 0.15926534334428127, 0.01408563358978433 ], [ -0.030400578900790887, 0.0863184516413344, 0.029238824807595102 ], [ -0.005515095323910174, 0.03412345060391818, 0.004057630638291695 ], ... ],
[ [ 0.04111762212014872, -0.01664709125213302, -0.02091611242137066 ], [ 0.09405434649358106, -0.038042238109388664, -0.08280755922930277 ], [ 0.008274587711135048, 0.0178313985384939, -0.022481551159748907 ], [ 0.041054957171629367, -0.01883742165944655, -0.03461997301470556 ], [ -0.06326651048766435, -0.037119965959455326, 0.0038149180672452498 ], [ 0.031377994227842536, -0.005763493278777557, -0.07891163111992125 ], [ 0.0023973016797112215, -0.056100426369127404, 0.03543732059802574 ], [ -0.03543795070961663, 0.04260100208864962, -0.009707772031241057 ], ... ],
[ [ 0.044376651450532184, 0.01675604540668277, -0.011050944633070912 ], [ -0.006031423040148233, 0.0893455824769077, -0.021484744635180648 ], [ 0.04051602252395516, 0.047313673499313515, 0.0640442021807481 ], [ -0.012277471419351572, -0.024173564612407616, -0.042656789365962094 ], [ -0.017157612437264724, -0.02630732714613332, 0.10264297100473113 ], [ -0.010761313462958482, 0.01898238720044971, -0.026334229769683803 ], [ -0.002081799799969978, 0.07614954918693867, -0.01615603335708194 ], [ 0.009780848314499664, -0.1025328962075354, 0.006463254249001187 ], ... ],
...
]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 30.66 seconds (0.289 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13616887855183
Reset training subject: 13617039500345
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=94.57237528495472}, derivative=-84.05354178633752}
New Minimum: 94.57237528495472 > 94.57237527654937
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=94.57237527654937}, derivative=-84.05354177886973}, evalInputDelta = -8.40535108181939E-9
New Minimum: 94.57237527654937 > 94.57237522611725
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=94.57237522611725}, derivative=-84.05354173406148}, evalInputDelta = -5.8837471783590445E-8
New Minimum: 94.57237522611725 > 94.57237487309236
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=94.57237487309236}, derivative=-84.05354142040554}, evalInputDelta = -4.11862359328552E-7
New Minimum: 94.57237487309236 > 94.57237240191827
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=94.57237240191827}, derivative=-84.05353922481322}, evalInputDelta = -2.8830364442455902E-6
New Minimum: 94.57237240191827 > 94.5723551037015
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=94.5723551037015}, derivative=-84.05352385566607}, evalInputDelta = -2.0181253219675455E-5
New Minimum: 94.5723551037015 > 94.57223401627252
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=94.57223401627252}, derivative=-84.05341627163772}, evalInputDelta = -1.4126868219932476E-4
New Minimum: 94.57223401627252 > 94.5713864086093
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=94.5713864086093}, derivative=-84.05266318344059}, evalInputDelta = -9.888763454171112E-4
New Minimum: 94.5713864086093 > 94.56545336760692
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=94.56545336760692}, derivative=-84.0473915661496}, evalInputDelta = -0.006921917347796125
New Minimum: 94.56545336760692 > 94.523932499958
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=94.523932499958}, derivative=-84.0104902495723}, evalInputDelta = -0.048442784996723276
New Minimum: 94.523932499958 > 94.23379697513782
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=94.23379697513782}, derivative=-83.75218125191816}, evalInputDelta = -0.33857830981689574
New Minimum: 94.23379697513782 > 92.22786508832465
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=92.22786508832465}, derivative=-81.94402896937582}, evalInputDelta = -2.3445101966300683
New Minimum: 92.22786508832465 > 79.41212998085996
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=79.41212998085996}, derivative=-69.28748733339361}, evalInputDelta = -15.160245304094758
New Minimum: 79.41212998085996 > 49.75378267357484
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=49.75378267357484}, derivative=19.282614422396918}, evalInputDelta = -44.81859261137988
49.75378267357484 <= 94.57237528495472
New Minimum: 49.75378267357484 > 47.26298425640721
F(1.1258491265882944) = LineSearchPoint{point=PointSample{avg=47.26298425640721}, derivative=0.004643487452863216}, evalInputDelta = -47.30939102854751
Right bracket at 1.1258491265882944
Converged to right
Fitness changed from 94.57237528495472 to 47.26298425640721
Iteration 1 complete. Error: 47.26298425640721 Total: 4.1568; Orientation: 0.0729; Line Search: 3.6325
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26298425640721}, derivative=-0.0026582764269896452}
New Minimum: 47.26298425640721 > 47.26148766894654
F(1.1258491265882944) = LineSearchPoint{point=PointSample{avg=47.26148766894654}, derivative=-3.168583099211797E-7}, evalInputDelta = -0.0014965874606645002
F(7.880943886118061) = LineSearchPoint{point=PointSample{avg=47.3153498329457}, derivative=0.01594743978565322}, evalInputDelta = 0.05236557653849161
F(0.6062264527783123) = LineSearchPoint{point=PointSample{avg=47.26180655729746}, derivative=-0.0012270674239234928}, evalInputDelta = -0.0011776991097462997
F(4.2435851694481865) = LineSearchPoint{point=PointSample{avg=47.272960734245785}, derivative=0.007360186371730658}, evalInputDelta = 0.00997647783857758
F(0.3264296284190913) = LineSearchPoint{point=PointSample{avg=47.262242297881265}, derivative=-0.001887625424020423}, evalInputDelta = -7.419585259427208E-4
F(2.2850073989336392) = LineSearchPoint{point=PointSample{avg=47.26307337641127}, derivative=0.0027362805292071397}, evalInputDelta = 8.912000406269271E-5
F(0.17576979991797226) = LineSearchPoint{point=PointSample{avg=47.262553480930336}, derivative=-0.0022433105019316942}, evalInputDelta = -4.3077547687175866E-4
F(1.230388599425806) = LineSearchPoint{point=PointSample{avg=47.26150053609197}, derivative=2.4648502968843217E-4}, evalInputDelta = -0.0014837203152353595
47.26150053609197 <= 47.26298425640721
New Minimum: 47.26148766894654 > 47.26148766892528
F(1.125983341031678) = LineSearchPoint{point=PointSample{avg=47.26148766892528}, derivative=1.6966006267595644E-12}, evalInputDelta = -0.0014965874819310443
Right bracket at 1.125983341031678
Converged to right
Fitness changed from 47.26298425640721 to 47.26148766892528
Iteration 2 complete. Error: 47.26148766892528 Total: 2.4586; Orientation: 0.0722; Line Search: 2.2363
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148766892528}, derivative=-3.9835368936035715E-7}
New Minimum: 47.26148766892528 > 47.26148732620028
F(1.125983341031678) = LineSearchPoint{point=PointSample{avg=47.26148732620028}, derivative=-2.1040309947166193E-7}, evalInputDelta = -3.427249950505029E-7
F(7.8818833872217455) = LineSearchPoint{point=PointSample{avg=47.261489714064155}, derivative=9.17300433496978E-7}, evalInputDelta = 2.04513887780422E-6
F(0.6062987220939804) = LineSearchPoint{point=PointSample{avg=47.26148745808392}, derivative=-2.971495255366102E-7}, evalInputDelta = -2.1084135681803673E-7
F(4.244091054657863) = LineSearchPoint{point=PointSample{avg=47.26148748159485}, derivative=3.1007545556077743E-7}, evalInputDelta = -1.8733042850271886E-7
47.26148748159485 <= 47.26148766892528
New Minimum: 47.26148732620028 > 47.26148719359447
F(2.386476250624132) = LineSearchPoint{point=PointSample{avg=47.26148719359447}, derivative=5.299003045564931E-16}, evalInputDelta = -4.753308076033136E-7
Right bracket at 2.386476250624132
Converged to right
Fitness changed from 47.26148766892528 to 47.26148719359447
Iteration 3 complete. Error: 47.26148719359447 Total: 1.6497; Orientation: 0.0719; Line Search: 1.4281
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148719359447}, derivative=-4.449639691045802E-7}
New Minimum: 47.26148719359447 > 47.26148672480286
F(2.386476250624132) = LineSearchPoint{point=PointSample{avg=47.26148672480286}, derivative=5.2090492603035376E-8}, evalInputDelta = -4.687916117518398E-7
47.26148672480286 <= 47.26148719359447
New Minimum: 47.26148672480286 > 47.26148671828897
F(2.1363774524896515) = LineSearchPoint{point=PointSample{avg=47.26148671828897}, derivative=7.986181321760206E-17}, evalInputDelta = -4.753054980710658E-7
Right bracket at 2.1363774524896515
Converged to right
Fitness changed from 47.26148719359447 to 47.26148671828897
Iteration 4 complete. Error: 47.26148671828897 Total: 1.1084; Orientation: 0.0717; Line Search: 0.8874
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148671828897}, derivative=-3.9857885875990454E-7}
New Minimum: 47.26148671828897 > 47.26148624819878
F(2.1363774524896515) = LineSearchPoint{point=PointSample{avg=47.26148624819878}, derivative=-4.1502730305616156E-8}, evalInputDelta = -4.700901925502876E-7
F(14.954642167427561) = LineSearchPoint{point=PointSample{avg=47.26149944749471}, derivative=2.1009540173406024E-6}, evalInputDelta = 1.272920573569536E-5
F(1.15035708980212) = LineSearchPoint{point=PointSample{avg=47.26148637037155}, derivative=-2.0630709714758665E-7}, evalInputDelta = -3.4791742109518964E-7
F(8.05249962861484) = LineSearchPoint{point=PointSample{avg=47.26148892767185}, derivative=9.473234658251649E-7}, evalInputDelta = 2.2093828775382462E-6
F(0.6194230483549876) = LineSearchPoint{point=PointSample{avg=47.261486503464766}, derivative=-2.950479101597542E-7}, evalInputDelta = -2.1482420464735696E-7
F(4.335961338484913) = LineSearchPoint{point=PointSample{avg=47.26148656
...skipping 6428 bytes...
nputDelta = 1.2729370581610056E-5
F(1.1480839696627365) = LineSearchPoint{point=PointSample{avg=47.2614835196098}, derivative=-2.0661507182692493E-7}, evalInputDelta = -3.477927137396364E-7
F(8.036587787639156) = LineSearchPoint{point=PointSample{avg=47.26148607726628}, derivative=9.492020083611511E-7}, evalInputDelta = 2.2098637657563813E-6
F(0.6181990605876274) = LineSearchPoint{point=PointSample{avg=47.26148365264778}, derivative=-2.955240784753427E-7}, evalInputDelta = -2.1475472777865434E-7
F(4.327393424113392) = LineSearchPoint{point=PointSample{avg=47.261483710724285}, derivative=3.2683896662188284E-7}, evalInputDelta = -1.566782259487809E-7
47.261483710724285 <= 47.26148386740251
New Minimum: 47.26148339752907 > 47.261483392397295
F(2.3794801253099442) = LineSearchPoint{point=PointSample{avg=47.261483392397295}, derivative=5.663420177577576E-16}, evalInputDelta = -4.7500521560550624E-7
Right bracket at 2.3794801253099442
Converged to right
Fitness changed from 47.26148386740251 to 47.261483392397295
Iteration 11 complete. Error: 47.261483392397295 Total: 2.0489; Orientation: 0.0714; Line Search: 1.8290
F(0.0) = LineSearchPoint{point=PointSample{avg=47.261483392397295}, derivative=-4.4582830399933254E-7}
New Minimum: 47.261483392397295 > 47.261482923874766
F(2.3794801253099442) = LineSearchPoint{point=PointSample{avg=47.261482923874766}, derivative=5.202586841165702E-8}, evalInputDelta = -4.6852252921780746E-7
47.261482923874766 <= 47.261483392397295
New Minimum: 47.261482923874766 > 47.26148291740648
F(2.130823938925042) = LineSearchPoint{point=PointSample{avg=47.26148291740648}, derivative=7.881360994537487E-17}, evalInputDelta = -4.749908129042524E-7
Right bracket at 2.130823938925042
Converged to right
Fitness changed from 47.261483392397295 to 47.26148291740648
Iteration 12 complete. Error: 47.26148291740648 Total: 1.1112; Orientation: 0.0713; Line Search: 0.8900
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148291740648}, derivative=-3.9947395760037994E-7}
New Minimum: 47.26148291740648 > 47.26148244758694
F(2.130823938925042) = LineSearchPoint{point=PointSample{avg=47.26148244758694}, derivative=-4.150057474003552E-8}, evalInputDelta = -4.6981953971680923E-7
F(14.915767572475295) = LineSearchPoint{point=PointSample{avg=47.26149564701289}, derivative=2.106339699173815E-6}, evalInputDelta = 1.2729606410744054E-5
F(1.1473667363442535) = LineSearchPoint{point=PointSample{avg=47.26148256964363}, derivative=-2.0671905899927143E-7}, evalInputDelta = -3.477628496284524E-7
F(8.031567154409775) = LineSearchPoint{point=PointSample{avg=47.26148512743825}, derivative=9.498103258576123E-7}, evalInputDelta = 2.2100317664808244E-6
F(0.6178128580315211) = LineSearchPoint{point=PointSample{avg=47.261482702668076}, derivative=-2.9568285831367954E-7}, evalInputDelta = -2.1473840661201393E-7
F(4.3246900062206475) = LineSearchPoint{point=PointSample{avg=47.2614827608306}, derivative=3.2706373544836065E-7}, evalInputDelta = -1.56575879373122E-7
47.2614827608306 <= 47.26148291740648
New Minimum: 47.26148244758694 > 47.26148244246098
F(2.3778546505003915) = LineSearchPoint{point=PointSample{avg=47.26148244246098}, derivative=5.653848572924705E-16}, evalInputDelta = -4.7494550159399296E-7
Right bracket at 2.3778546505003915
Converged to right
Fitness changed from 47.26148291740648 to 47.26148244246098
Iteration 13 complete. Error: 47.26148244246098 Total: 2.1521; Orientation: 0.0705; Line Search: 1.9327
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148244246098}, derivative=-4.4604469158634316E-7}
New Minimum: 47.26148244246098 > 47.2614819739808
F(2.3778546505003915) = LineSearchPoint{point=PointSample{avg=47.2614819739808}, derivative=5.2008677517241244E-8}, evalInputDelta = -4.6848018087075616E-7
47.2614819739808 <= 47.26148244246098
New Minimum: 47.2614819739808 > 47.26148196752379
F(2.1295497832462424) = LineSearchPoint{point=PointSample{avg=47.26148196752379}, derivative=7.808921148701523E-17}, evalInputDelta = -4.7493718824398456E-7
Right bracket at 2.1295497832462424
Converged to right
Fitness changed from 47.26148244246098 to 47.26148196752379
Iteration 14 complete. Error: 47.26148196752379 Total: 1.2374; Orientation: 0.0716; Line Search: 1.0155
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148196752379}, derivative=-3.9969582403769736E-7}
New Minimum: 47.26148196752379 > 47.26148149774451
F(2.1295497832462424) = LineSearchPoint{point=PointSample{avg=47.26148149774451}, derivative=-4.1504744725233895E-8}, evalInputDelta = -4.6977928036540106E-7
F(14.906848482723698) = LineSearchPoint{point=PointSample{avg=47.26149469756912}, derivative=2.107641707987567E-6}, evalInputDelta = 1.2730045327202788E-5
F(1.1466806525172075) = LineSearchPoint{point=PointSample{avg=47.26148161978169}, derivative=-2.068237042704841E-7}, evalInputDelta = -3.477421017805682E-7
F(8.026764567620452) = LineSearchPoint{point=PointSample{avg=47.26148417774634}, derivative=9.504090076076166E-7}, evalInputDelta = 2.210222547205376E-6
F(0.6174434282784963) = LineSearchPoint{point=PointSample{avg=47.261481752796286}, derivative=-2.9584160566157104E-7}, evalInputDelta = -2.1472750688644737E-7
F(4.322103997949474) = LineSearchPoint{point=PointSample{avg=47.26148181103744}, derivative=3.272837026431438E-7}, evalInputDelta = -1.5648635098841623E-7
47.26148181103744 <= 47.26148196752379
New Minimum: 47.26148149774451 > 47.261481492623695
F(2.37630752398817) = LineSearchPoint{point=PointSample{avg=47.261481492623695}, derivative=5.633404373951538E-16}, evalInputDelta = -4.749000979131779E-7
Right bracket at 2.37630752398817
Converged to right
Fitness changed from 47.26148196752379 to 47.261481492623695
Iteration 15 complete. Error: 47.261481492623695 Total: 2.0956; Orientation: 0.0761; Line Search: 1.8655
F(0.0) = LineSearchPoint{point=PointSample{avg=47.261481492623695}, derivative=-4.4626161922964236E-7}
New Minimum: 47.261481492623695 > 47.26148102417101
F(2.37630752398817) = LineSearchPoint{point=PointSample{avg=47.26148102417101}, derivative=5.199220653615383E-8}, evalInputDelta = -4.6845268286688224E-7
47.26148102417101 <= 47.261481492623695
New Minimum: 47.26148102417101 > 47.26148101772489
F(2.128342600907612) = LineSearchPoint{point=PointSample{avg=47.26148101772489}, derivative=7.718815371959386E-17}, evalInputDelta = -4.748988047253988E-7
Right bracket at 2.128342600907612
Converged to right
Fitness changed from 47.261481492623695 to 47.26148101772489
Iteration 16 complete. Error: 47.26148101772489 Total: 1.2445; Orientation: 0.0721; Line Search: 1.0228
F(0.0) = LineSearchPoint{point=PointSample{avg=47.26148101772489}, derivative=-3.99916820444458E-7}
New Minimum: 47.26148101772489 > 47.26148054797027
F(2.128342600907612) = LineSearchPoint{point=PointSample{avg=47.26148054797027}, derivative=-4.151081889042783E-8}, evalInputDelta = -4.6975461742704283E-7
F(14.898398206353285) = LineSearchPoint{point=PointSample{avg=47.26149374846841}, derivative=2.108925167404616E-6}, evalInputDelta = 1.27307435207058E-5
F(1.146030631257945) = LineSearchPoint{point=PointSample{avg=47.261480669992956}, derivative=-2.069289733171837E-7}, evalInputDelta = -3.4773193391401946E-7
F(8.022214418805614) = LineSearchPoint{point=PointSample{avg=47.26148322817101}, derivative=9.509981027598599E-7}, evalInputDelta = 2.210446119477183E-6
F(0.6170934168312011) = LineSearchPoint{point=PointSample{avg=47.26148080300196}, derivative=-2.960002873363337E-7}, evalInputDelta = -2.1472293099122908E-7
F(4.319653917818408) = LineSearchPoint{point=PointSample{avg=47.26148086131474}, derivative=3.2749890937242996E-7}, evalInputDelta = -1.5641015238543332E-7
47.26148086131474 <= 47.26148101772489
New Minimum: 47.26148054797027 > 47.26148054285393
F(2.374848644899729) = LineSearchPoint{point=PointSample{avg=47.26148054285393}, derivative=5.60205409638688E-16}, evalInputDelta = -4.7487095855558437E-7
Right bracket at 2.374848644899729
Converged to right
Fitness changed from 47.26148101772489 to 47.26148054285393
Iteration 17 complete. Error: 47.26148054285393 Total: 2.0408; Orientation: 0.0712; Line Search: 1.8205
Final threshold in iteration 17: 47.26148054285393 (> 0.0) after 30.657s (< 30.000s)
Returns
47.26148054285393
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[-0.008543839288533178, 0.0033604881198859874, -0.007139481716540529, -0.0053499449396725, 0.008736447128523311, -0.014633613048307622, -0.007773325436574161, -0.009504566233285952, -0.007522389869488133, -4.0862256939800214E-4, -0.0036787755129261573, -0.006234352598256956, 0.011908789913568009, -0.004289337672396667, 0.008680332902123097, 0.0028466714470160967, -0.007243005967650044, 0.005999000612214235, 0.011672155534209876, 1.8585605114316235E-4, -5.328687947379324E-4, -0.0011627396536150537, 0.001275156314381936, 0.00401517865052155, 0.010118758156842687, 4.528524770514974E-4, -0.013128402946922307, -0.006127783259523781, -0.010214303329126989, -0.00344150048616551, -0.002214336619623713, -0.02006241704296704, 0.00881141601852254, -0.010993759722906216, -7.175565518803688E-4, -7.163576879369724E-4, -0.010228724957729923, 0.00757119340041071, -0.010177635588691268, -0.012379329877372584, 0.006959380834372882, -0.004559188146408392, -0.01046665683804752, 0.015308829557787493, 0.0035949123123116935, -0.0018053373800661814, 0.009217424097715145, -0.009506412345621048, -0.0018689659921293537, 0.006976084774733713, -0.0128328377307612, -0.012863044314728568, 0.009341165178799562, -0.01593475026833247, 0.0021768980156251254, 0.0063542905729735075, 0.008204044972074666, 0.0029147803808681257, -0.0015105954997682426, -5.718938651429531E-4, 0.0028976993193392062, 0.007956229546784465, 0.005484376704917964, -0.008467070785505439, 0.005558384404020949, 0.00361857014720323, -7.410054700627707E-4, -0.001239093000623532, 0.004177985903280362, 0.0010785178702698006, 0.017760120606841286, -0.008991603035892916, -0.006065913865589356, -0.005023584010588172, -0.0035502316632163126, -0.0011837064872003934, 0.004280085194446536, -0.005058644261613375, 0.0035787259351826414, -0.011760905669596937, 0.0022983644265073895]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.5440581200583755, 0.5239852318458211, 1.0000212780695332 ], [ -1.4799402662893466, 2.0343890939328747E-6, 1.3200120872583776 ], [ -1.8279014813518453, 1.1399520438512343, 0.12404455124182483 ], [ -1.7119862008257585, 0.29603426191050897, 1.7398542478873618 ], [ 0.5639655561421472, 1.8479655435154811, -1.3599812466870012 ], [ 0.7839288984805035, -0.8359214657100906, -0.17202823848064794 ], [ -0.9840311733024787, -0.7800289198964122, 0.02004806512973749 ], [ 1.3559712670996735, 0.16403103674335226, -0.05606554070533383 ], ... ],
[ [ -1.6798868508167695, -1.5200379328074882, -0.2040200024136578 ], [ -1.3838835884693317, 0.3559316673226759, 1.9719592121180207 ], [ -0.019948464887155754, -0.6519885859061233, -1.97989470066061 ], [ 0.6799329519199263, 0.8918687872312796, -0.7479655253594628 ], [ 0.024050842824828108, 1.33998385060881, 0.4719669628480335 ], [ 0.739890570373346, 0.972034695435102, 0.8079870367430806 ], [ -0.779994015025711, -0.10001972813409256, -1.647906542337582 ], [ 0.7439765623011153, 0.5680059361269154, 1.779958648220746 ], ... ],
[ [ 0.5160289874501479, 1.6599777247729162, 1.1640348741612365 ], [ -1.851902880232004, -0.8039614590723537, 1.6158303119856146 ], [ -0.18794111195421392, 1.8160522313533165, 0.183999984071692 ], [ 1.5200003229223, -0.22791345712908223, 0.8759780495201825 ], [ 1.551968949728697, -0.7000691126018614, 1.287981630442645 ], [ -0.876076948396546, -0.8040455852821571, -0.7799102851859522 ], [ 1.7399498195462606, -0.33195568359258243, 0.9920289213715066 ], [ -1.8120090282204622, -1.3399849743202639, -0.964004564751006 ], ... ],
[ [ -0.8999445304220716, -0.8760348659246422, 1.563972691200512 ], [ 0.6760084750870712, -1.4718879600661892, 1.647863106985142 ], [ -1.07985812611128, 1.5119342413245316, 1.6318722854277385 ], [ 1.5118521251929569, -0.27592946255907586, -0.8719303609684553 ], [ -1.6159870617807646, 0.6599522439856705, 1.2160161647316756 ], [ -0.23997084319670173, 1.5920292595083916, -0.21993334348752047 ], [ 1.391996906806399, -0.7360870771093178, -0.9518381152483237 ], [ 1.1680414715731045, -0.0800486777951165, -0.6000333943661625 ], ... ],
[ [ 0.9880781315955609, 0.2720172449883746, 1.3520275137646014 ], [ 1.679843330087682, -1.7920289239938285, 0.13211876670156328 ], [ -0.019946657407392054, -0.12802688438409654, 1.4759124650798034 ], [ 1.97592861449608, -0.1080029551407615, 0.36411056429032196 ], [ 0.7360163397859963, -1.6279842679232923, -0.4720416039506232 ], [ 1.9159529499057868, -1.160092627416026, -0.9559297756344183 ], [ -1.6558949945353023, -0.8039383609094326, -0.7319670842212082 ], [ 0.5759261720576276, 1.8920060608234248, -0.21993556145727924 ], ... ],
[ [ -1.5519621377563029, 1.0000005470798286, 1.4359927653390847 ], [ -1.3440092060754296, 0.5039944078655283, -1.1838889964181223 ], [ 1.1199231756107852, 0.07595368597061537, -0.9599500118601797 ], [ 0.37194317742400307, -0.9439544168507131, 1.051958635289835 ], [ -1.4360397064512564, 1.711960848058967, 0.180009698536373 ], [ -1.0719385722509756, 1.3640223803541907, 1.9279159205466634 ], [ -0.19589183231509313, -0.14393556775002822, 1.731762991084953 ], [ -1.2519980136878668, -0.952013592920335, -1.5319700916871377 ], ... ],
[ [ -1.9759031794017612, -1.5399937569359532, 0.2839368874419612 ], [ -0.43584701700718254, -0.9559285043704425, -0.6840757632557056 ], [ -1.487926251628219, -0.9959669110314542, 0.5039610907356008 ], [ 1.6399360330738073, -0.10399419347490796, 1.4840434258248918 ], [ -1.8358934823707243, 0.4799859092717918, 1.4798523879895218 ], [ -0.6439511920535029, 0.38799516803745543, -1.6840028684459474 ], [ -1.484008066026734, -0.8399874169652746, 1.1758912208249952 ], [ -0.028048943329062808, -1.2399144745555692, -0.6279258061921377 ], ... ],
[ [ 0.6079868408602056, 0.05994585950481834, -0.759939132949995 ], [ -1.235994774529459, 1.9118624358166447, -0.26785603709639255 ], [ -1.8079066815843166, 0.4599869608246325, 1.8999653384326447 ], [ 1.6839391490942022, 1.0079950517663578, -1.9317970814848777 ], [ 1.2078467737872316, -1.012068594773056, -1.1798659030180643 ], [ 1.815939566287976, 1.395904794899207, 1.8721528964190697 ], [ -1.7559742789928385, 0.29195375164199505, -1.9959804186637178 ], [ 1.7919161137858426, -1.827900693832405, -1.7839475175400226 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.02 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 0.02487073571385323, 0.05118896279943765, -0.03474274450298251 ], [ -0.03672923813560925, 0.12665778982921033, 0.007474882109906617 ], [ -0.001090910835560254, 0.08062845234274774, -0.038727680501864 ], [ -0.019116794247727425, 0.07081784951193223, 0.014800486626831392 ], [ 0.023536727402649164, -0.042497870775384715, -0.049335964576547535 ], [ -0.007027996876449714, -0.029997512075915833, 0.05455660811416301 ], [ 0.02151989798145035, -0.049876632772272854, -0.0023890163619494325 ], [ 0.00816370952707306, -0.02301125045247438, -0.0501158733448279 ], ... ],
[ [ 0.04712006419217556, 0.03161073104464845, -0.0413116612803877 ], [ 0.05045999859487309, -0.09861000253004784, -0.0544143965519937 ], [ 0.099032417197577, -0.0796879328898024, -0.044171943916934645 ], [ 0.03784866761559395, -0.09617968524229827, 0.008138734762248403 ], [ -0.04831718395582107, 0.013802479716277282, 0.06710100146203446 ], [ 4.5737798945934E-5, 0.044445720656287616, -4.474477013807212E-4 ], [ -0.04607843086969653, 0.07949351834747193, 0.03857427761912962 ], [ -0.009778942862698432, -0.020946093486642746, 0.023358344969215166 ], ... ],
[ [ 0.019710102773667564, 0.03203537391043926, -0.04351892483366685 ], [ 0.03329015554433554, 0.08230073960134111, 0.0174807624929821 ], [ -0.009474978714814732, -0.03008932687598765, -0.021712220538906733 ], [ 0.06694392721228225, -0.005349632608934339, 0.035026694817784726 ], [ 0.009693071708014906, -0.0689830256640732, -0.018776192938437842 ], [ 0.02708397910289039, -0.018866712659474434, -0.010417500714070204 ], [ -0.013750745122544993, -0.07887070066160656, -0.003965104236772985 ], [ -0.02109797280942462, 0.04057973932498634, -0.0013689127587820477 ], ... ],
[ [ 0.019089948923729328, 9.605982074256168E-4, -0.00700538931857207 ], [ 0.022108300014858424, -0.05298807050470414, -0.04927939593951394 ], [ -0.038217659989726364, -0.054670808891877504, -1.567782840017935E-4 ], [ -0.021203052117756197, 0.02185353786985771, -0.0081335239236929 ], [ -0.1329147250117985, 0.04358550193246873, 0.057997569212563924 ], [ -0.06397608797228647, 0.024795745247274577, 0.033856864094133916 ], [ -0.017680256656955867, 0.0039268938218449305, -0.005297915346911712 ], [ -0.02330119590862438, -0.07289339045097137, -0.005874731611160281 ], ... ],
[ [ 0.05276884612710133, -0.07143282930542715, -0.019656709774168016 ], [ -0.001990626650397771, -0.09825153461693323, -0.02228216496282884 ], [ 0.0031743674571190887, -0.07015350353125123, 0.02970361882125759 ], [ -0.016111995694957155, -0.057847898416946986, -0.06992686352221955 ], [ 0.09412067405254469, -0.13749805006311416, 0.002385268866195685 ], [ 0.09363212128488745, -0.11262389629163323, -0.04413270191302565 ], [ -0.028868189737110046, 0.0324694757516571, 0.03513728225299102 ], [ 0.022838495051602237, -0.010880815625161983, 0.017391919033872712 ], ... ],
[ [ -0.04213714690823764, 0.09265438411350552, 0.014360244480687633 ], [ -0.04235981141701055, 0.0863415369367517, -0.015664776568328223 ], [ -0.019291058753888105, -0.04309651344317766, 0.016796932788298146 ], [ -0.022463007597589485, 0.04062889286038125, 0.01146083953797021 ], [ -0.07569183707634936, 0.0651413750863527, -0.0035289859451245443 ], [ 0.022322429013894127, 0.1581185058281121, 0.013957410690869244 ], [ -0.029935616147759882, 0.08529347510234818, 0.02941973092726246 ], [ -0.005442630107949685, 0.0337821014949441, 0.004659753672686559 ], ... ],
[ [ 0.040868569006368254, -0.0162643800341078, -0.020362712720199058 ], [ 0.093275128859104, -0.03753134505972521, -0.08243760217022392 ], [ 0.008536270856965691, 0.01831140816142567, -0.02205207294377882 ], [ 0.041445787844515976, -0.01895507258922598, -0.03347275145985793 ], [ -0.06297227677852918, -0.03706999119594336, 0.0035876274954485298 ], [ 0.030985826299437066, -0.00534844120718952, -0.07873891357718704 ], [ 0.002840243088859207, -0.05558303167553656, 0.035427302762110875 ], [ -0.03448347118214307, 0.04210378158179108, -0.009606949177210496 ], ... ],
[ [ 0.043800432015690866, 0.016998431503984356, -0.011509508208695396 ], [ -0.006184718315594118, 0.08899345978875581, -0.022117707073705212 ], [ 0.04017770389996284, 0.046737859939290076, 0.06328994247610664 ], [ -0.012466775313295955, -0.02401888999082911, -0.042593670146078746 ], [ -0.01707244744001059, -0.026181039639034728, 0.10155715533574013 ], [ -0.010908310761337477, 0.018247777100453114, -0.027071031152185194 ], [ -0.0024328593337957765, 0.07607731292872608, -0.016112516823805073 ], [ 0.009950562305223867, -0.10221393298063647, 0.007079230696629665 ], ... ],
...
]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 33.16 seconds (0.242 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 13647597087116
Reset training subject: 13647745693453
Adding measurement 3fe61f51 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 94.57237528495472 < 94.57237528495472. Total: 1
th(0)=94.57237528495472;dx=-84.05354178633752
Adding measurement 66a8bbfc to history. Total: 1
New Minimum: 94.57237528495472 > 86.74823548046467
WOLF (strong): th(2.154434690031884)=86.74823548046467; dx=76.76554836696447 evalInputDelta=7.824139804490045
Adding measurement 4dcbf5ac to history. Total: 2
New Minimum: 86.74823548046467 > 47.35102751490329
END: th(1.077217345015942)=47.35102751490329; dx=-3.6254676375286494 evalInputDelta=47.22134777005142
Fitness changed from 94.57237528495472 to 47.35102751490329
Iteration 1 complete. Error: 47.35102751490329 Total: 1.2922; Orientation: 0.0996; Line Search: 0.7441
Non-optimal measurement 47.35102751490329 < 47.35102751490329. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 47.35102751490329 < 47.35102751490329. Total: 3
th(0)=47.35102751490329;dx=-0.1588224204384863
Non-optimal measurement 47.36177727876125 < 47.35102751490329. Total: 3
Armijo: th(2.3207944168063896)=47.36177727876125; dx=0.1680861893480153 evalInputDelta=-0.010749763857958783
Adding measurement 62f3dc46 to history. Total: 3
New Minimum: 47.35102751490329 > 47.26156643728305
WOLF (strong): th(1.1603972084031948)=47.26156643728305; dx=0.004631957022192299 evalInputDelta=0.08946107762024269
Non-optimal measurement 47.30013248762287 < 47.26156643728305. Total: 4
END: th(0.3867990694677316)=47.30013248762287; dx=-0.1043376118247544 evalInputDelta=0.05089502728042561
Fitness changed from 47.35102751490329 to 47.26156643728305
Iteration 2 complete. Error: 47.26156643728305 Total: 1.3167; Orientation: 0.0854; Line Search: 1.0814
Non-optimal measurement 47.26156643728305 < 47.26156643728305. Total: 4
Rejected: LBFGS Orientation magnitude: 1.333e-02, gradient 1.185e-02, dot -1.000; [f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26156643728305, 47.35102751490329, 86.74823548046467, 94.57237528495472
LBFGS Accumulation History: 3 points
Removed measurement 62f3dc46 to history. Total: 3
Adding measurement 6a3ca6a4 to history. Total: 3
th(0)=47.26156643728305;dx=-1.4040099591331256E-4
Adding measurement 7295dd60 to history. Total: 4
New Minimum: 47.26156643728305 > 47.261492699002034
END: th(0.8333333333333335)=47.261492699002034; dx=-3.657087856207661E-5 evalInputDelta=7.373828101719937E-5
Fitness changed from 47.26156643728305 to 47.261492699002034
Iteration 3 complete. Error: 47.261492699002034 Total: 2.9844; Orientation: 2.3478; Line Search: 0.4871
Non-optimal measurement 47.261492699002034 < 47.261492699002034. Total: 5
Rejected: LBFGS Orientation magnitude: 3.495e-03, gradient 3.105e-03, dot -1.000; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.261492699002034, 47.26156643728305, 47.35102751490329, 86.74823548046467, 94.57237528495472
Rejected: LBFGS Orientation magnitude: 3.497e-03, gradient 3.105e-03, dot -1.000; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.261492699002034, 47.26156643728305, 47.35102751490329, 86.74823548046467
LBFGS Accumulation History: 3 points
Removed measurement 7295dd60 to history. Total: 4
Removed measurement 6a3ca6a4 to history. Total: 3
Adding measurement 3334b51 to history. Total: 3
th(0)=47.261492699002034;dx=-9.643135058399481E-6
Adding measurement 71afc774 to history. Total: 4
New Minimum: 47.261492699002034 > 47.26148889544319
WOLF (strong): th(1.7953622416932369)=47.26148889544319; dx=5.40604156978444E-6 evalInputDelta=3.803558840331789E-6
Adding measurement 1ba53ed to history. Total: 5
New Minimum: 47.26148889544319 > 47.26148741988218
END: th(0.8976811208466184)=47.26148741988218; dx=-2.1185467418407765E-6 evalInputDelta=5.279119854151304E-6
Fitness changed from 47.261492699002034 to 47.26148741988218
Iteration 4 complete. Error: 47.26148741988218 Total: 6.1924; Orientation: 5.1544; Line Search: 0.8888
Non-optimal measurement 47.26148741988218 < 47.26148741988218. Total: 6
Rejected: LBFGS Orientation magnitude: 8.841e-04, gradient 7.723e-04, dot -1.000; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148741988218, 47.26148889544319, 47.261492699002034, 47.35102751490329, 86.74823548046467, 94.57237528495472
Rejected: LBFGS Orientation magnitude: 8.843e-04, gradient 7.723e-04, dot -1.000; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148741988218, 47.26148889544319, 47.261492699002034, 47.35102751490329, 86.74823548046467
Rejected: LBFGS Orientation magnitude: 8.856e-04, gradient 7.723e-04, dot -1.000; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148741988218, 47.26148889544319, 47.261492699002034, 47.35102751490329
LBFGS Accumulation History: 3 points
Removed measurement 1ba53ed to history. Total: 5
Removed measurement 71afc774 to history. Total: 4
Removed measurement 3334b51 to history. Total: 3
Adding measurement 4d2458 to history. Total: 3
th(0)=47.26148741988218;dx=-5.964392640497723E-7
Adding measurement ed2dcf1 to history. Total: 4
New Minimum: 47.26148741988218 > 47.26148690810365
WOLF (strong): th(1.9339953473386584)=47.26148690810365; dx=6.719442919679247E-8 evalInputDelta=5.117785306651967E-7
Non-optimal measurement 47.26148700355985 < 47.26148690810365. Total: 5
END: th(0.9669976736693292)=47.26148700355985; dx=-2.646224173228984E-7 evalInputDelta=4.1632232949950776E-7
Fitness changed from 47.26148741988218 to 47.26148690810365
Iteration 5 complete. Error: 47.26148690810365 Total: 9.2887; Orientation: 8.3821; Line Search: 0.7336
Non-optimal measurement 47.26148690810365 < 47.26148690810365. Total: 5
Rejected: LBFGS Orientation magnitude: 1.190e-03, gradient 6.399e-04, dot -0.942; [05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148690810365, 47.26148741988218, 47.35102751490329, 86.74823548046467, 94.57237528495472
Rejected: LBFGS Orientation magnitude: 1.190e-03, gradient 6.399e-04, dot -0.942; [f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148690810365, 47.26148741988218, 47.35102751490329, 86.74823548046467
LBFGS Accumulation History: 3 points
Removed measurement ed2dcf1 to history. Total: 4
Removed measurement 4d2458 to history. Total: 3
Adding measurement 79d4cea5 to history. Total: 3
th(0)=47.26148690810365;dx=-4.0948652050352633E-7
Adding measurement 23e0d07a to history. Total: 4
New Minimum: 47.26148690810365 > 47.26148643919406
END: th(2.083333333333334)=47.26148643919406; dx=-4.066669630696155E-8 evalInputDelta=4.689095902676854E-7
Fitness changed from 47.26148690810365 to 47.26148643919406
Iteration 6 complete. Error: 47.26148643919406 Total: 5.9388; Orientation: 5.2834; Line Search: 0.5052
Non-optimal measurement 47.26148643919406 < 47.26148643919406. Total: 5
Rejected: LBFGS Orientation magnitude: 1.654e-03, gradient 5.955e-04, dot -0.906; [f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148643919406, 47.26148690810365, 47.35102751490329, 86.74823548046467, 94.57237528495472
Rejected: LBFGS Orientation magnitude: 1.654e-03, gradient 5.955e-04, dot -0.906; [f208267c-3309-49a4-9875-063f805504d2 = 1.000/1.000e+00, 05ba629e-5c6a-42e6-84e4-9addc6970743 = 1.000/1.000e+00, f52ecc4b-2939-4e51-957e-fd7fa659b3af = 1.000/1.000e+00]
Orientation rejected. Popping history element from 47.26148643919406, 47.26148690810365, 47.35102751490329, 86.74823548046467
LBFGS Accumulation History: 3 points
Removed measurement 23e0d07a to history. Total: 4
Removed measurement 79d4cea5 to history. Total: 3
Adding measurement 19b3148b to history. Total: 3
th(0)=47.26148643919406;dx=-3.5464001680278E-7
Adding measurement 7b783308 to history. Total: 4
New Minimum: 47.26148643919406 > 47.26148613949037
WOLF (strong): th(4.488405604233092)=47.26148613949037; dx=2.2109429881918854E-7 evalInputDelta=2.997036858687352E-7
Adding measurement 4abb79f3 to history. Total: 5
New Minimum: 47.26148613949037 > 47.26148596632608
END: th(2.244202802116546)=47.26148596632608; dx=-6.677285877126279E-8 evalInputDelta=4.728679812160408E-7
Fitness changed from 47.26148643919406 to 47.26148596632608
Iteration 7 complete. Error: 47.26148596632608 Total: 6.1503; Orientation: 5.2479; Line Search: 0.7534
Final threshold in iteration 7: 47.26148596632608 (> 0.0) after 33.164s (< 30.000s)
Returns
47.26148596632608
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
[-0.008615496002608497, 0.0033151658386733566, -0.007118681291019045, -0.00542739476158205, 0.008869620124916383, -0.01465147159020932, -0.007888937803606765, -0.009363629241850874, -0.007579213209305526, -3.8569342553849306E-4, -0.0036134207125202546, -0.0061790645697697415, 0.01190035418057737, -0.004127759254566422, 0.008632210156240978, 0.0027172105287294017, -0.007225463520692889, 0.00597088058713034, 0.011832957344793586, 2.770773156974245E-4, -5.433852610663548E-4, -0.001032465943342903, 0.0011766568400835037, 0.004143277190547828, 0.010210893047396804, 4.472431477135013E-4, -0.013167049821592794, -0.005980063972798234, -0.010236483842637378, -0.003352224457529048, -0.002202183441483154, -0.01995118430356637, 0.008840430136114638, -0.010929365094428833, -6.473289381800871E-4, -6.1955315101438E-4, -0.010092483516034548, 0.007474098083896917, -0.010136729230367613, -0.01232321011776156, 0.00695628237065434, -0.004555188295860241, -0.010655968527680406, 0.015270487609566284, 0.0034796337294904773, -0.0018629253177559135, 0.009136970157324635, -0.009513226251627838, -0.001912924032209353, 0.007065658613165608, -0.012782874643819527, -0.012908850612304457, 0.009174022766064011, -0.015772599042441205, 0.0020516120942434264, 0.006270990854903545, 0.008110687732722643, 0.00273674234568712, -0.001547340301262207, -6.975856762794633E-4, 0.0027898563652380536, 0.008151575887412065, 0.005433211116714249, -0.008561401968807895, 0.005609803593429355, 0.0034705600048504006, -8.295875414345849E-4, -0.0011189779516399508, 0.00427081120005492, 0.0010284955008130884, 0.017862649988670015, -0.009002873011025172, -0.00588424206239325, -0.005002877542095063, -0.0033845430676032458, -0.001146908104845126, 0.004080365759018118, -0.005020762942435628, 0.0035908056541082184, -0.011907454153264153, 0.0024337282554046527]
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 0.5440584752260532, 0.5239794963649832, 1.0000189069144856 ], [ -1.4799334907473658, -2.2722765355908554E-6, 1.3200107164654609 ], [ -1.8279039682535412, 1.139953159429497, 0.12404137771189418 ], [ -1.7119767333865228, 0.2960383565111016, 1.7398556902596276 ], [ 0.5639655522386285, 1.8479648323854876, -1.3599840165585686 ], [ 0.7839345191370256, -0.8359351046637356, -0.17202171040660807 ], [ -0.9840281573284074, -0.7800269878346606, 0.02004404297466262 ], [ 1.355974002032519, 0.1640302012788931, -0.05606764449747248 ], ... ],
[ [ -1.6798919574398539, -1.5200338533847888, -0.20401066432411696 ], [ -1.3838747368878654, 0.3559467421170492, 1.9719587586610072 ], [ -0.019946436566837884, -0.6519865668112208, -1.9798899186488261 ], [ 0.6799367006831988, 0.8918768570451987, -0.7479644506605888 ], [ 0.024044372666675976, 1.3399782169033974, 0.4719798871780238 ], [ 0.7398990234839953, 0.9720413040954445, 0.8079856471137635 ], [ -0.780004415109452, -0.10001683077613799, -1.647902702412113 ], [ 0.7439831335099513, 0.5680089066982859, 1.7799709591352086 ], ... ],
[ [ 0.5160402894943248, 1.6599779133194776, 1.1640262985610679 ], [ -1.851898697381047, -0.8039712068137193, 1.61583115762026 ], [ -0.1879390382072897, 1.816046093244467, 0.18399422377063826 ], [ 1.5200057237812044, -0.22792095328779638, 0.8759733813581889 ], [ 1.5519696197573538, -0.700075972922771, 1.2879847836194052 ], [ -0.8760762786134039, -0.8040479782730453, -0.7799120248414381 ], [ 1.7399532192350768, -0.33196074566930844, 0.9920370064597923 ], [ -1.8119965756801428, -1.3399801852134972, -0.9640231948277599 ], ... ],
[ [ -0.8999502740932628, -0.8760253631431547, 1.5639784595452124 ], [ 0.6760134564609986, -1.4718816862689634, 1.647873035357129 ], [ -1.0798672496630222, 1.511940794363803, 1.6318738452796517 ], [ 1.5118636372948444, -0.2759291345819961, -0.8719316725768581 ], [ -1.615980463061292, 0.6599398153121091, 1.2160086111895754 ], [ -0.23996158524289213, 1.5920305892439688, -0.21993865759677558 ], [ 1.3920077179133317, -0.7360738789631199, -0.9518495855820777 ], [ 1.168043022040381, -0.08004574425284754, -0.6000232368553087 ], ... ],
[ [ 0.9880745774873617, 0.27200550411979674, 1.352025699140009 ], [ 1.6798498267109079, -1.7920307068352046, 0.13210691856640366 ], [ -0.0199444837195178, -0.12803958754282127, 1.4759304219199205 ], [ 1.9759371931698706, -0.10799380440193411, 0.36410282785027204 ], [ 0.7360134540706358, -1.6279740482758274, -0.472033429089873 ], [ 1.9159587271616931, -1.1600829108068764, -0.9559278326465229 ], [ -1.655888320379571, -0.8039457364833016, -0.7319714487128234 ], [ 0.5759414757954073, 1.8920031161563264, -0.21994527980899375 ], ... ],
[ [ -1.5519606379555244, 1.0000003183497366, 1.4359945734563773 ], [ -1.3440040994525901, 0.5039949563153946, -1.1838914763080517 ], [ 1.1199348491216508, 0.07595608200513868, -0.9599534285512304 ], [ 0.3719422752675043, -0.9439646480707462, 1.0519473323461457 ], [ -1.436027482746507, 1.711958687553501, 0.18001029648249448 ], [ -1.0719415587004677, 1.364028535237241, 1.9279123526502153 ], [ -0.19589478118283013, -0.14393076153265014, 1.7317760750375513 ], [ -1.252001351523807, -0.9520034699886492, -1.5319755985825059 ], ... ],
[ [ -1.9759041694300157, -1.539996237240429, 0.2839365374089502 ], [ -0.4358480680857189, -0.9559309681736715, -0.6840757062118544 ], [ -1.487917618686664, -0.9959698815128621, 0.5039629960598678 ], [ 1.6399345558540679, -0.10399003679508356, 1.4840591604530295 ], [ -1.8358996605190248, 0.48000284775619867, 1.479860438305993 ], [ -0.6439616950790672, 0.3880023790552975, -1.6839918851529296 ], [ -1.4839980676876212, -0.8399893959618671, 1.1758911382358714 ], [ -0.028054017153519507, -1.239927899265235, -0.6279225417474399 ], ... ],
[ [ 0.6079848452936071, 0.05995687913953815, -0.7599386872308665 ], [ -1.236000780496308, 1.911867433614138, -0.2678530563777318 ], [ -1.8079096611696155, 0.45998863806656465, 1.8999648785971068 ], [ 1.6839388503493795, 1.0079957815384244, -1.9318099975978498 ], [ 1.2078633760613093, -1.0120765693973783, -1.1798750244835383 ], [ 1.8159470223561074, 1.395893691102788, 1.8721493240988216 ], [ -1.755972906330889, 0.2919590788267354, -1.9959872196358328 ], [ 1.7919135177048213, -1.8279109822616617, -1.7839484380635482 ], ... ],
...
]
To produce the following output:
TrainingTester.java:633 executed in 0.03 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[
[ [ 0.02565765263087257, 0.05089737208023887, -0.03382474508773342 ], [ -0.03640562086843072, 0.1265092750223192, 0.0077389145589433625 ], [ -0.0012266200170611777, 0.08004529847466696, -0.03899851901141373 ], [ -0.01988351577320772, 0.0709696461050157, 0.014052592158028572 ], [ 0.023577026641619737, -0.042748554622221886, -0.04980417845652158 ], [ -0.007230690805423456, -0.02991114580031362, 0.05455880513106615 ], [ 0.02125753452727486, -0.04993930471769344, -0.0023189517002296565 ], [ 0.008416751204049605, -0.022691969929829272, -0.05063500879987391 ], ... ],
[ [ 0.04745674571811175, 0.030747001427048386, -0.04043667674461146 ], [ 0.050066904419953796, -0.09859445262597635, -0.054660031778342784 ], [ 0.0993844715833115, -0.08027761557730297, -0.04327449637558631 ], [ 0.03859932833131252, -0.09643934613898296, 0.008339791234914355 ], [ -0.04829765320461133, 0.013711425866040653, 0.06763214821002116 ], [ 6.353545894214711E-5, 0.044447226270702994, -5.515833633090583E-4 ], [ -0.045598450674583076, 0.07886212949161286, 0.038948067719404254 ], [ -0.009841249559434806, -0.020522667507909105, 0.023482566126035893 ], ... ],
[ [ 0.020622307592163307, 0.03166819573219486, -0.04231024941425046 ], [ 0.03429532135786924, 0.08169855873549116, 0.017989987434178182 ], [ -0.009377909976843827, -0.029924082960861173, -0.022626749558080885 ], [ 0.0672213394513952, -0.006101731410000839, 0.034484263596973104 ], [ 0.009879037021024842, -0.06824480038453096, -0.01914558870555043 ], [ 0.02700492106337481, -0.019298332916041692, -0.010763532089596172 ], [ -0.014767714362202682, -0.07827105265177714, -0.004637643959330383 ], [ -0.020867588180560375, 0.040133487795324836, -0.0013480153585268915 ], ... ],
[ [ 0.018614564453566296, 8.341654687443544E-4, -0.006962345798142995 ], [ 0.02230932359167963, -0.05341864769035847, -0.0499489572180441 ], [ -0.03821060436376987, -0.0538248948230875, -3.1296705693133875E-4 ], [ -0.020954754156431978, 0.02127318089891139, -0.008469143631884569 ], [ -0.13343577888967906, 0.044208464683522575, 0.057777516299217846 ], [ -0.0645049759283863, 0.0250912291303148, 0.033534362060882936 ], [ -0.017678327949422413, 0.004466872728933141, -0.005379740553038617 ], [ -0.023171176242374896, -0.07150765636690419, -0.006189696637975616 ], ... ],
[ [ 0.05302032607836925, -0.07112725961796901, -0.01949438953951587 ], [ -0.0015815677260868426, -0.09806303238532807, -0.022317914440191478 ], [ 0.003156327202393993, -0.0694353446043743, 0.029753080351836065 ], [ -0.016218633942929373, -0.057781169545424865, -0.07023757170807356 ], [ 0.09493420070692574, -0.13698345368694897, 0.0026948091857183884 ], [ 0.09432431032704279, -0.11266675910801237, -0.043227901853800245 ], [ -0.028733377031220555, 0.031505900814203715, 0.03532721065547548 ], [ 0.022841702739280538, -0.010989760155548213, 0.017813609922544143 ], ... ],
[ [ -0.04215864712605463, 0.09257846440250375, 0.014965795196663667 ], [ -0.04217790446912678, 0.0857946075333366, -0.015013773554412786 ], [ -0.018595654031003578, -0.04211218695140927, 0.01728414519800362 ], [ -0.021622430870612157, 0.040910849029741354, 0.012196035043888115 ], [ -0.07533673629502946, 0.06504980424300916, -0.0038400631016816864 ], [ 0.022434281900484406, 0.15726958132610797, 0.01406230450129608 ], [ -0.030367904352104246, 0.08513660452993912, 0.028674931978644406 ], [ -0.005434900562811753, 0.03342341927635326, 0.003834000276455852 ], ... ],
[ [ 0.04034232921745986, -0.016219715672073205, -0.020577535041022582 ], [ 0.09310375912116269, -0.037842739099894246, -0.08225210078583305 ], [ 0.007859423635340736, 0.01774965580612282, -0.022671340309952645 ], [ 0.04136239726992669, -0.018990160160120003, -0.035007499707359266 ], [ -0.06373881251911417, -0.036555609302484623, 0.003802743265282148 ], [ 0.03097138535479766, -0.005981281281335652, -0.07865610729816211 ], [ 0.0014957689936739862, -0.05528762091525181, 0.03520758614048819 ], [ -0.03567147027273758, 0.041711660746684184, -0.009700628343526524 ], ... ],
[ [ 0.04456211303753135, 0.016440218026285015, -0.01051894944961302 ], [ -0.005375656351865983, 0.0881970090461599, -0.020605214895344174 ], [ 0.040204386312722676, 0.04659663528588005, 0.06377074612846577 ], [ -0.012318871629316297, -0.02456561798208776, -0.042470437700132785 ], [ -0.016678248216296456, -0.02597428953948906, 0.10254039379724379 ], [ -0.010870524720758307, 0.019139054987132507, -0.02565853677938338 ], [ -0.00216737569951434, 0.07606454019312058, -0.01571265905823671 ], [ 0.009423767548189002, -0.10139892047020121, 0.006277873093640622 ], ... ],
...
]
TrainingTester.java:432 executed in 0.01 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, 1.6745070284857149], [33.0, 1.6753294076098477]; valueStats=DoubleSummaryStatistics{count=57, sum=2694.084970, min=47.261449, average=47.264649, max=47.351028}
Plotting 33 points for GD
Plotting 17 points for CjGD
Plotting 7 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.01 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, 1.6745070284857149], [31.871, 1.6753294076098477]; valueStats=DoubleSummaryStatistics{count=57, sum=2694.084970, min=47.261449, average=47.264649, max=47.351028}
Plotting 33 points for GD
Plotting 17 points for CjGD
Plotting 7 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": 3.3268686456608094 }, "CjGD": { "type": "NonConverged", "value": 0.20106863337445702 }, "GD": { "type": "NonConverged", "value": 0.04399527467744724 } }, "model":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": 47.26148596632608 }, "CjGD": { "type": "NonConverged", "value": 47.26148054285393 }, "GD": { "type": "NonConverged", "value": 47.26144854810845 } }}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": 3.3268686456608094 }, "CjGD": { "type": "NonConverged", "value": 0.20106863337445702 }, "GD": { "type": "NonConverged", "value": 0.04399527467744724 } }, "model":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "complete":{ "LBFGS": { "type": "NonConverged", "value": 47.26148596632608 }, "CjGD": { "type": "NonConverged", "value": 47.26148054285393 }, "GD": { "type": "NonConverged", "value": 47.26144854810845 } }} | OK |
{
"result": "OK",
"performance": {
"execution_time": "212.033",
"gc_time": "2.800"
},
"created_on": 1586748101097,
"file_name": "trainingTest",
"report": {
"simpleName": "NoPadding",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.NoPadding",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": 3.3268686456608094
},
"CjGD": {
"type": "NonConverged",
"value": 0.20106863337445702
},
"GD": {
"type": "NonConverged",
"value": 0.04399527467744724
}
},
"model": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
},
"complete": {
"LBFGS": {
"type": "NonConverged",
"value": 47.26148596632608
},
"CjGD": {
"type": "NonConverged",
"value": 47.26148054285393
},
"GD": {
"type": "NonConverged",
"value": 47.26144854810845
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/NoPadding/trainingTest/202004132141",
"id": "08a22366-66d9-40fc-ac7e-82b47f36cdb5",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ConvolutionLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
"javaDoc": ""
}
}