Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.02 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ -1.476 ], [ 1.912 ], [ 1.876 ], [ -1.688 ], [ -0.128 ], [ -1.028 ] ],
[ [ 1.512 ], [ -1.492 ], [ 0.636 ], [ 1.612 ], [ 1.208 ], [ 1.64 ] ],
[ [ 0.048 ], [ -1.228 ], [ 1.764 ], [ 1.556 ], [ -0.556 ], [ -0.768 ] ],
[ [ 1.556 ], [ 1.704 ], [ 0.392 ], [ 0.092 ], [ 1.032 ], [ -0.712 ] ],
[ [ -0.408 ], [ 1.108 ], [ 1.356 ], [ 0.7 ], [ 1.048 ], [ -0.852 ] ],
[ [ -0.176 ], [ -1.616 ], [ 1.552 ], [ 0.496 ], [ 0.08 ], [ -0.636 ] ],
[ [ -0.384 ], [ 0.788 ], [ 0.028 ], [ -0.068 ], [ -0.608 ], [ 1.524 ] ],
[ [ -1.516 ], [ -1.72 ], [ -0.384 ], [ -1.572 ], [ -0.804 ], [ 0.3 ] ]
]
[
[ [ 1.552 ], [ 0.3 ], [ -0.408 ], [ 0.028 ], [ -1.688 ], [ -0.128 ] ],
[ [ -1.72 ], [ 1.556 ], [ -0.068 ], [ 1.764 ], [ -1.476 ], [ -1.516 ] ],
[ [ 0.048 ], [ 1.032 ], [ 1.208 ], [ -0.556 ], [ 0.496 ], [ -1.572 ] ],
[ [ 1.612 ], [ -0.384 ], [ -1.492 ], [ -0.636 ], [ -1.028 ], [ 0.092 ] ],
[ [ -0.852 ], [ 1.912 ], [ -0.712 ], [ 1.108 ], [ 1.512 ], [ 1.524 ] ],
[ [ 0.636 ], [ 1.356 ], [ -0.176 ], [ -0.384 ], [ -0.608 ], [ 0.7 ] ],
[ [ 1.048 ], [ 1.876 ], [ 1.556 ], [ 0.788 ], [ -1.228 ], [ 0.08 ] ],
[ [ -1.616 ], [ -0.804 ], [ 0.392 ], [ 1.64 ], [ -0.768 ], [ 1.704 ] ]
]
[
[ [ 0.028 ], [ 0.392 ], [ 1.612 ], [ 0.636 ], [ -0.068 ], [ 1.512 ] ],
[ [ -0.712 ], [ -0.176 ], [ -0.852 ], [ 0.3 ], [ -0.768 ], [ -1.688 ] ],
[ [ 1.208 ], [ -1.72 ], [ 0.788 ], [ 0.496 ], [ 1.356 ], [ -0.636 ] ],
[ [ 1.048 ], [ 1.876 ], [ 1.556 ], [ 1.108 ], [ 0.092 ], [ -1.028 ] ],
[ [ -1.492 ], [ -1.476 ], [ -0.384 ], [ 1.704 ], [ 1.552 ], [ -0.384 ] ],
[ [ 1.912 ], [ 1.032 ], [ 1.64 ], [ -0.128 ], [ 1.764 ], [ 0.08 ] ],
[ [ -0.608 ], [ -0.556 ], [ -0.408 ], [ -1.228 ], [ 0.7 ], [ 1.524 ] ],
[ [ -1.616 ], [ -1.516 ], [ -1.572 ], [ -0.804 ], [ 0.048 ], [ 1.556 ] ]
]
[
[ [ 0.028 ], [ -1.476 ], [ 1.64 ], [ -0.408 ], [ 1.032 ], [ 0.08 ] ],
[ [ -1.492 ], [ 0.7 ], [ -1.616 ], [ 0.3 ], [ -1.688 ], [ -0.852 ] ],
[ [ 1.208 ], [ -0.804 ], [ -1.028 ], [ -1.72 ], [ 1.612 ], [ -0.128 ] ],
[ [ -0.068 ], [ -0.556 ], [ 1.764 ], [ 1.524 ], [ 1.556 ], [ -0.176 ] ],
[ [ 0.092 ], [ -0.384 ], [ -0.608 ], [ -0.636 ], [ 1.108 ], [ -1.516 ] ],
[ [ 1.512 ], [ -0.712 ], [ 0.392 ], [ 1.556 ], [ 1.048 ], [ 0.496 ] ],
[ [ -1.572 ], [ 0.636 ], [ 1.552 ], [ 1.704 ], [ 0.048 ], [ 0.788 ] ],
[ [ -0.768 ], [ -0.384 ], [ -1.228 ], [ 1.356 ], [ 1.912 ], [ 1.876 ] ]
]
[
[ [ 1.512 ], [ 1.764 ], [ 0.496 ], [ -1.516 ], [ 1.64 ], [ 1.108 ] ],
[ [ 1.356 ], [ 1.612 ], [ 0.3 ], [ -1.572 ], [ 1.912 ], [ 1.048 ] ],
[ [ 0.048 ], [ -0.556 ], [ 1.704 ], [ 1.556 ], [ -0.852 ], [ -0.636 ] ],
[ [ -1.72 ], [ 0.08 ], [ -1.492 ], [ -0.384 ], [ -0.068 ], [ -0.768 ] ],
[ [ 0.392 ], [ -0.128 ], [ 0.636 ], [ -0.608 ], [ -0.176 ], [ -1.028 ] ],
[ [ -1.476 ], [ -0.804 ], [ 1.552 ], [ -0.384 ], [ -1.228 ], [ 0.7 ] ],
[ [ -1.688 ], [ 0.092 ], [ -0.712 ], [ 1.208 ], [ -1.616 ], [ -0.408 ] ],
[ [ 1.524 ], [ 0.028 ], [ 0.788 ], [ 1.876 ], [ 1.032 ], [ 1.556 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 1.20 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2002968324890
Reset training subject: 2003017261576
Constructing line search parameters: GD
th(0)=2.494658133333333;dx=-0.16631054222222222
New Minimum: 2.494658133333333 > 2.1492186844466117
WOLFE (weak): th(2.154434690031884)=2.1492186844466117; dx=-0.15436703550616995 evalInputDelta=0.3454394488867214
New Minimum: 2.1492186844466117 > 1.829510740749581
END: th(4.308869380063768)=1.829510740749581; dx=-0.1424235287901177 evalInputDelta=0.6651473925837521
Fitness changed from 2.494658133333333 to 1.829510740749581
Iteration 1 complete. Error: 1.829510740749581 Total: 0.1678; Orientation: 0.0046; Line Search: 0.0783
th(0)=1.829510740749581;dx=-0.12196738271663875
New Minimum: 1.829510740749581 > 0.8724463643026089
END: th(9.283177667225559)=0.8724463643026089; dx=-0.08422588660447031 evalInputDelta=0.9570643764469721
Fitness changed from 1.829510740749581 to 0.8724463643026089
Iteration 2 complete. Error: 0.8724463643026089 Total: 0.0611; Orientation: 0.0028; Line Search: 0.0409
th(0)=0.8724463643026089;dx=-0.05816309095350726
New Minimum: 0.8724463643026089 > 0.09693848492251204
END: th(20.000000000000004)=0.09693848492251204; dx=-0.019387696984502413 evalInputDelta=0.7755078793800969
Fitness changed from 0.8724463643026089 to 0.09693848492251204
Iteration 3 complete. Error: 0.09693848492251204 Total: 0.0493; Orientation: 0.0016; Line Search: 0.0338
th(0)=0.09693848492251204;dx=-0.006462565661500803
New Minimum: 0.09693848492251204 > 0.01845212270699858
WOLF (strong): th(43.088693800637685)=0.01845212270699858; dx=0.0028195514369966515 evalInputDelta=0.07848636221551347
New Minimum: 0.01845212270699858 > 0.007701016129902769
END: th(21.544346900318843)=0.007701016129902769; dx=-0.0018215071122520757 evalInputDelta=0.08923746879260927
Fitness changed from 0.09693848492251204 to 0.007701016129902769
Iteration 4 complete. Error: 0.007701016129902769 Total: 0.0718; Orientation: 0.0020; Line Search: 0.0550
th(0)=0.007701016129902769;dx=-5.134010753268513E-4
New Minimum: 0.007701016129902769 > 0.0023058672556143887
WOLF (strong): th(46.4158883361278)=0.0023058672556143887; dx=2.809311574737843E-4 evalInputDelta=0.00539514887428838
New Minimum: 0.0023058672556143887 > 3.947371653259078E-4
END: th(23.2079441680639)=3.947371653259078E-4; dx=-1.1623495892653354E-4 evalInputDelta=0.007306278964576861
Fitness changed from 0.007701016129902769 to 3.947371653259078E-4
Iteration 5 complete. Error: 3.947371653259078E-4 Total: 0.0652; Orientation: 0.0015; Line Search: 0.0531
th(0)=3.947371653259078E-4;dx=-2.6315811021727188E-5
New Minimum: 3.947371653259078E-4 > 1.7543874014484804E-4
WOLF (strong): th(50.000000000000014)=1.7543874014484804E-4; dx=1.7543874014484797E-5 evalInputDelta=2.1929842518105975E-4
New Minimum: 1.7543874014484804E-4 > 1.0964921259052962E-5
END: th(25.000000000000007)=1.0964921259052962E-5; dx=-4.385968503621191E-6 evalInputDelta=3.8377224406685483E-4
Fitness changed from 3.947371653259078E-4 to 1.0964921259052962E-5
Iteration 6 complete. Error: 1.0964921259052962E-5 Total: 0.0586; Orientation: 0.0015; Line Search: 0.0477
th(0)=1.0964921259052962E-5;dx=-7.309947506035308E-7
New Minimum: 1.0964921259052962E-5 > 6.936421200671884E-6
WOLF (strong): th(53.860867250797114)=6.936421200671884E-6; dx=5.814056235060102E-7 evalInputDelta=4.028500058381077E-6
New Minimum: 6.936421200671884E-6 > 1.1479343863612316E-7
END: th(26.930433625398557)=1.1479343863612316E-7; dx=-7.479456354875648E-8 evalInputDelta=1.0850127820416838E-5
Fitness changed from 1.0964921259052962E-5 to 1.1479343863612316E-7
Iteration 7 complete. Error: 1.1479343863612316E-7 Total: 0.0575; Orientation: 0.0017; Line Search: 0.0440
th(0)=1.1479343863612316E-7;dx=-7.652895909074878E-9
New Minimum: 1.1479343863612316E-7 > 1.0013974726789528E-7
WOLF (strong): th(58.01986042015976)=1.0013974726789528E-7; dx=7.147769172743191E-9 evalInputDelta=1.4653691368227878E-8
New Minimum: 1.0013974726789528E-7 > 1.2502768043043146E-10
END: th(29.00993021007988)=1.2502768043043146E-10; dx=-2.525633681655847E-10 evalInputDelta=1.1466841095569273E-7
Fitness changed from 1.1479343863612316E-7 to 1.2502768043043146E-10
Iteration 8 complete. Error: 1.2502768043043146E-10 Total: 0.0547; Orientation: 0.0013; Line Search: 0.0428
Low gradient: 2.8870709543345307E-6
th(0)=1.2502768043043146E-10;dx=-8.335178695362096E-12
Armijo: th(62.50000000000003)=1.4673387494984542E-10; dx=9.029776919983046E-12 evalInputDelta=-2.170619451941396E-11
New Minimum: 1.2502768043043146E-10 > 2.170619451854501E-13
WOLF (strong): th(31.250000000000014)=2.170619451854501E-13; dx=3.472991123017371E-13 evalInputDelta=1.24810618485246E-10
END: th(10.416666666666671)=5.327664854766152E-11; dx=-5.441019426141651E-12 evalInputDelta=7.175103188276993E-11
Fitness changed from 1.2502768043043146E-10 to 2.170619451854501E-13
Iteration 9 complete. Error: 2.170619451854501E-13 Total: 0.0526; Orientation: 0.0013; Line Search: 0.0435
Low gradient: 1.2029462309553437E-7
th(0)=2.170619451854501E-13;dx=-1.4470796345696673E-14
New Minimum: 2.170619451854501E-13 > 1.3776907295310191E-14
END: th(22.442028021165466)=1.3776907295310191E-14; dx=-3.64566244317798E-15 evalInputDelta=2.032850378901399E-13
Fitness changed from 2.170619451854501E-13 to 1.3776907295310191E-14
Iteration 10 complete. Error: 1.3776907295310191E-14 Total: 0.0307; Orientation: 0.0013; Line Search: 0.0209
Low gradient: 3.0306113019554536E-8
th(0)=1.3776907295310191E-14;dx=-9.18460486354013E-16
New Minimum: 1.3776907295310191E-14 > 5.1543731727564894E-15
WOLF (strong): th(48.34988368346647)=5.1543731727564894E-15; dx=5.617881030602198E-16 evalInputDelta=8.6225341225537E-15
New Minimum: 5.1543731727564894E-15 > 5.194093441538124E-16
END: th(24.174941841733236)=5.194093441538124E-16; dx=-1.7833619166213932E-16 evalInputDelta=1.3257497951156378E-14
Fitness changed from 1.3776907295310191E-14 to 5.194093441538124E-16
Iteration 11 complete. Error: 5.194093441538124E-16 Total: 0.0588; Orientation: 0.0016; Line Search: 0.0449
Low gradient: 5.8844956971905555E-9
th(0)=5.194093441538124E-16;dx=-3.4627289610254163E-17
New Minimum: 5.194093441538124E-16 > 2.814469227930981E-16
WOLF (strong): th(52.083333333333364)=2.814469227930981E-16; dx=2.548953263004437E-17 evalInputDelta=2.379624213607143E-16
New Minimum: 2.814469227930981E-16 > 9.042572011406734E-18
END: th(26.041666666666682)=9.042572011406734E-18; dx=-4.568878490107898E-18 evalInputDelta=5.103667721424056E-16
Fitness changed from 5.194093441538124E-16 to 9.042572011406734E-18
Iteration 12 complete. Error: 9.042572011406734E-18 Total: 0.0538; Orientation: 0.0013; Line Search: 0.0439
Low gradient: 7.764265155787651E-10
th(0)=9.042572011406734E-18;dx=-6.028381340937823E-19
New Minimum: 9.042572011406734E-18 > 6.8469821532848484E-18
WOLF (strong): th(56.105070052913675)=6.8469821532848484E-18; dx=5.245710594037548E-19 evalInputDelta=2.195589858121886E-18
New Minimum: 6.8469821532848484E-18 > 3.8105597342991585E-20
END: th(28.052535026456837)=3.8105597342991585E-20; dx=-3.913353821704718E-20 evalInputDelta=9.004466414063743E-18
Fitness changed from 9.042572011406734E-18 to 3.8105597342991585E-20
Iteration 13 complete. Error: 3.8105597342991585E-20 Total: 0.0601; Orientation: 0.0016; Line Search: 0.0481
Zero gradient: 5.0402114600475236E-11
th(0)=3.8105597342991585E-20;dx=-2.5403731561994387E-21
Armijo: th(60.4373546043331)=3.922473529483755E-20; dx=2.577407801157712E-21 evalInputDelta=-1.1191379518459652E-21
New Minimum: 3.8105597342991585E-20 > 2.0246802488454433E-24
WOLF (strong): th(30.21867730216655)=2.0246802488454433E-24; dx=1.851745725684341E-23 evalInputDelta=3.810357266274274E-20
END: th(10.072892434055516)=1.6812593465537382E-20; dx=-1.687409476602869E-21 evalInputDelta=2.1293003877454202E-20
Fitness changed from 3.8105597342991585E-20 to 2.0246802488454433E-24
Iteration 14 complete. Error: 2.0246802488454433E-24 Total: 0.0618; Orientation: 0.0011; Line Search: 0.0515
Zero gradient: 3.6739445185843907E-13
th(0)=2.0246802488454433E-24;dx=-1.349786832563629E-25
New Minimum: 2.0246802488454433E-24 > 1.5492107487048022E-25
END: th(21.701388888888903)=1.5492107487048022E-25; dx=-3.7337240063737623E-26 evalInputDelta=1.8697591739749632E-24
Fitness changed from 2.0246802488454433E-24 to 1.5492107487048022E-25
Iteration 15 complete. Error: 1.5492107487048022E-25 Total: 0.0350; Orientation: 0.0014; Line Search: 0.0245
Zero gradient: 1.0162712068159766E-13
th(0)=1.5492107487048022E-25;dx=-1.0328071658032015E-26
New Minimum: 1.5492107487048022E-25 > 4.832243494034366E-26
WOLF (strong): th(46.75422504409473)=4.832243494034366E-26; dx=5.7681744149386006E-27 evalInputDelta=1.0659863993013657E-25
New Minimum: 4.832243494034366E-26 > 7.550028398908242E-27
END: th(23.377112522047366)=7.550028398908242E-27; dx=-2.280017734184738E-27 evalInputDelta=1.4737104647157199E-25
Fitness changed from 1.5492107487048022E-25 to 7.550028398908242E-27
Iteration 16 complete. Error: 7.550028398908242E-27 Total: 0.0477; Orientation: 0.0012; Line Search: 0.0374
Zero gradient: 2.2435133754758025E-14
th(0)=7.550028398908242E-27;dx=-5.0333522659388275E-28
New Minimum: 7.550028398908242E-27 > 3.477812936225092E-27
WOLF (strong): th(50.364462170277584)=3.477812936225092E-27; dx=3.4161426585785936E-28 evalInputDelta=4.0722154626831496E-27
New Minimum: 3.477812936225092E-27 > 1.948803895354576E-28
END: th(25.182231085138792)=1.948803895354576E-28; dx=-8.086587874964586E-29 evalInputDelta=7.355148009372784E-27
Fitness changed from 7.550028398908242E-27 to 1.948803895354576E-28
Iteration 17 complete. Error: 1.948803895354576E-28 Total: 0.0432; Orientation: 0.0013; Line Search: 0.0321
Zero gradient: 3.604445306705389E-15
th(0)=1.948803895354576E-28;dx=-1.2992025969030507E-29
New Minimum: 1.948803895354576E-28 > 1.2736145592542383E-28
WOLF (strong): th(54.253472222222264)=1.2736145592542383E-28; dx=1.0502860821672762E-29 evalInputDelta=6.751893361003377E-29
New Minimum: 1.2736145592542383E-28 > 1.7674162803144445E-30
END: th(27.126736111111132)=1.7674162803144445E-30; dx=-1.2365562138248414E-30 evalInputDelta=1.9311297325514316E-28
Fitness changed from 1.948803895354576E-28 to 1.7674162803144445E-30
Iteration 18 complete. Error: 1.7674162803144445E-30 Total: 0.0451; Orientation: 0.0011; Line Search: 0.0360
Zero gradient: 3.4326047255832265E-16
th(0)=1.7674162803144445E-30;dx=-1.1782775202096297E-31
New Minimum: 1.7674162803144445E-30 > 1.5756837352019162E-30
WOLF (strong): th(58.44278130511842)=1.5756837352019162E-30; dx=1.111668698102457E-31 evalInputDelta=1.9173254511252838E-31
New Minimum: 1.5756837352019162E-30 > 2.1586464793340265E-34
END: th(29.22139065255921)=2.1586464793340265E-34; dx=-5.100504469385529E-34 evalInputDelta=1.7672004156665113E-30
Fitness changed from 1.7674162803144445E-30 to 2.1586464793340265E-34
Iteration 19 complete. Error: 2.1586464793340265E-34 Total: 0.0392; Orientation: 0.0011; Line Search: 0.0310
Zero gradient: 3.79354405917389E-18
th(0)=2.1586464793340265E-34;dx=-1.4390976528893514E-35
Armijo: th(62.955577712846996)=2.2007761968674974E-34; dx=1.4524721663920405E-35 evalInputDelta=-4.2129717533470955E-36
New Minimum: 2.1586464793340265E-34 > 0.0
END: th(31.477788856423498)=0.0; dx=0.0 evalInputDelta=2.1586464793340265E-34
Fitness changed from 2.1586464793340265E-34 to 0.0
Iteration 20 complete. Error: 0.0 Total: 0.0443; Orientation: 0.0012; Line Search: 0.0292
Zero gradient: 0.0
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0270; Orientation: 0.0009; Line Search: 0.0189
Iteration 21 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 21
Final threshold in iteration 21: 0.0 (> 0.0) after 1.186s (< 30.000s)
Returns
0.0
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.31 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2004162102598
Reset training subject: 2004169306985
Constructing line search parameters: GD
F(0.0) = LineSearchPoint{point=PointSample{avg=2.494658133333333}, derivative=-0.16631054222222222}
New Minimum: 2.494658133333333 > 2.494658133316702
F(1.0E-10) = LineSearchPoint{point=PointSample{avg=2.494658133316702}, derivative=-0.16631054222166786}, evalInputDelta = -1.6631140908884845E-11
New Minimum: 2.494658133316702 > 2.494658133216916
F(7.000000000000001E-10) = LineSearchPoint{point=PointSample{avg=2.494658133216916}, derivative=-0.16631054221834163}, evalInputDelta = -1.1641709818377421E-10
New Minimum: 2.494658133216916 > 2.4946581325184116
F(4.900000000000001E-9) = LineSearchPoint{point=PointSample{avg=2.4946581325184116}, derivative=-0.16631054219505817}, evalInputDelta = -8.149214636432589E-10
New Minimum: 2.4946581325184116 > 2.494658127628882
F(3.430000000000001E-8) = LineSearchPoint{point=PointSample{avg=2.494658127628882}, derivative=-0.16631054203207385}, evalInputDelta = -5.704451133681232E-9
New Minimum: 2.494658127628882 > 2.4946580934021725
F(2.4010000000000004E-7) = LineSearchPoint{point=PointSample{avg=2.4946580934021725}, derivative=-0.16631054089118352}, evalInputDelta = -3.993116060030388E-8
New Minimum: 2.4946580934021725 > 2.494657853815213
F(1.6807000000000003E-6) = LineSearchPoint{point=PointSample{avg=2.494657853815213}, derivative=-0.1663105329049513}, evalInputDelta = -2.795181202053243E-7
New Minimum: 2.494657853815213 > 2.4946561767068185
F(1.1764900000000001E-5) = LineSearchPoint{point=PointSample{avg=2.4946561767068185}, derivative=-0.16631047700132562}, evalInputDelta = -1.9566265145876116E-6
New Minimum: 2.4946561767068185 > 2.4946444369638447
F(8.235430000000001E-5) = LineSearchPoint{point=PointSample{avg=2.4946444369638447}, derivative=-0.16631008567594596}, evalInputDelta = -1.3696369488336302E-5
New Minimum: 2.4946444369638447 > 2.4945622595364862
F(5.764801000000001E-4) = LineSearchPoint{point=PointSample{avg=2.4945622595364862}, derivative=-0.16630734639828854}, evalInputDelta = -9.587379684683484E-5
New Minimum: 2.4945622595364862 > 2.493987055444312
F(0.004035360700000001) = LineSearchPoint{point=PointSample{avg=2.493987055444312}, derivative=-0.16628817145468625}, evalInputDelta = -6.710778890210989E-4
New Minimum: 2.493987055444312 > 2.4899624838666243
F(0.028247524900000005) = LineSearchPoint{point=PointSample{avg=2.4899624838666243}, derivative=-0.1661539468494704}, evalInputDelta = -0.004695649466708751
New Minimum: 2.4899624838666243 > 2.4618814791318804
F(0.19773267430000002) = LineSearchPoint{point=PointSample{avg=2.4618814791318804}, derivative=-0.16521437461295946}, evalInputDelta = -0.032776654201452704
New Minimum: 2.4618814791318804 > 2.2697732651332356
F(1.3841287201) = LineSearchPoint{point=PointSample{avg=2.2697732651332356}, derivative=-0.15863736895738284}, evalInputDelta = -0.22488486820009745
New Minimum: 2.2697732651332356 > 1.143497905226171
F(9.688901040700001) = LineSearchPoint{point=PointSample{avg=1.143497905226171}, derivative=-0.11259832936834654}, evalInputDelta = -1.3511602281071622
F(67.8223072849) = LineSearchPoint{point=PointSample{avg=3.965195151965733}, derivative=0.2096749477549075}, evalInputDelta = 1.4705370186323998
F(5.217100560376924) = LineSearchPoint{point=PointSample{avg=1.7024436991727754}, derivative=-0.13738858145475072}, evalInputDelta = -0.7922144341605577
New Minimum: 1.143497905226171 > 0.11782142648010355
F(36.51970392263847) = LineSearchPoint{point=PointSample{avg=0.11782142648010355}, derivative=0.03614318315007844}, evalInputDelta = -2.3768367068532297
0.11782142648010355 <= 2.494658133333333
New Minimum: 0.11782142648010355 > 2.598741533396764E-32
F(30.0) = LineSearchPoint{point=PointSample{avg=2.598741533396764E-32}, derivative=-1.413298490583592E-18}, evalInputDelta = -2.494658133333333
Left bracket at 30.0
Converged to left
Fitness changed from 2.494658133333333 to 2.598741533396764E-32
Iteration 1 complete. Error: 2.598741533396764E-32 Total: 0.2595; Orientation: 0.0010; Line Search: 0.2336
Zero gradient: 4.162324297310149E-17
F(0.0) = LineSearchPoint{point=PointSample{avg=2.598741533396764E-32}, derivative=-1.732494355597843E-33}
New Minimum: 2.598741533396764E-32 > 0.0
F(30.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}, evalInputDelta = -2.598741533396764E-32
0.0 <= 2.598741533396764E-32
Converged to right
Fitness changed from 2.598741533396764E-32 to 0.0
Iteration 2 complete. Error: 0.0 Total: 0.0299; Orientation: 0.0011; Line Search: 0.0216
Zero gradient: 0.0
F(0.0) = LineSearchPoint{point=PointSample{avg=0.0}, derivative=0.0}
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.0212; Orientation: 0.0011; Line Search: 0.0108
Iteration 3 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 3
Final threshold in iteration 3: 0.0 (> 0.0) after 0.311s (< 30.000s)
Returns
0.0
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 6.61 seconds (0.118 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 2004479404638
Reset training subject: 2004488189352
Adding measurement 520dfeea to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD
Non-optimal measurement 2.494658133333333 < 2.494658133333333. Total: 1
th(0)=2.494658133333333;dx=-0.16631054222222222
Adding measurement 5a747276 to history. Total: 1
New Minimum: 2.494658133333333 > 2.1492186844466117
WOLFE (weak): th(2.154434690031884)=2.1492186844466117; dx=-0.15436703550616998 evalInputDelta=0.3454394488867214
Adding measurement 1898ba0 to history. Total: 2
New Minimum: 2.1492186844466117 > 1.829510740749581
END: th(4.308869380063768)=1.829510740749581; dx=-0.1424235287901177 evalInputDelta=0.6651473925837521
Fitness changed from 2.494658133333333 to 1.829510740749581
Iteration 1 complete. Error: 1.829510740749581 Total: 0.0665; Orientation: 0.0068; Line Search: 0.0328
Non-optimal measurement 1.829510740749581 < 1.829510740749581. Total: 3
LBFGS Accumulation History: 3 points
Non-optimal measurement 1.829510740749581 < 1.829510740749581. Total: 3
th(0)=1.829510740749581;dx=-0.12196738271663875
Adding measurement 36f5837e to history. Total: 3
New Minimum: 1.829510740749581 > 0.8724463643026089
END: th(9.283177667225559)=0.8724463643026089; dx=-0.08422588660447031 evalInputDelta=0.9570643764469721
Fitness changed from 1.829510740749581 to 0.8724463643026089
Iteration 2 complete. Error: 0.8724463643026089 Total: 0.0319; Orientation: 0.0023; Line Search: 0.0215
Non-optimal measurement 0.8724463643026089 < 0.8724463643026089. Total: 4
Rejected: LBFGS Orientation magnitude: 7.235e+00, gradient 2.412e-01, dot -1.000; [9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.8724463643026089, 1.829510740749581, 2.1492186844466117, 2.494658133333333
LBFGS Accumulation History: 3 points
Removed measurement 36f5837e to history. Total: 3
Adding measurement 10601e11 to history. Total: 3
th(0)=0.8724463643026089;dx=-0.05816309095350726
Adding measurement 63b01358 to history. Total: 4
New Minimum: 0.8724463643026089 > 0.09693848492251204
END: th(20.000000000000004)=0.09693848492251204; dx=-0.019387696984502413 evalInputDelta=0.7755078793800969
Fitness changed from 0.8724463643026089 to 0.09693848492251204
Iteration 3 complete. Error: 0.09693848492251204 Total: 0.1138; Orientation: 0.0842; Line Search: 0.0202
Non-optimal measurement 0.09693848492251204 < 0.09693848492251204. Total: 5
Rejected: LBFGS Orientation magnitude: 2.412e+00, gradient 8.039e-02, dot -1.000; [9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.09693848492251204, 0.8724463643026089, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 2.412e+00, gradient 8.039e-02, dot -1.000; [9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.09693848492251204, 0.8724463643026089, 1.829510740749581, 2.1492186844466117
LBFGS Accumulation History: 3 points
Removed measurement 63b01358 to history. Total: 4
Removed measurement 10601e11 to history. Total: 3
Adding measurement c10f241 to history. Total: 3
th(0)=0.09693848492251204;dx=-0.006462565661500803
Adding measurement 59408f4a to history. Total: 4
New Minimum: 0.09693848492251204 > 0.01845212270699858
WOLF (strong): th(43.088693800637685)=0.01845212270699858; dx=0.0028195514369966515 evalInputDelta=0.07848636221551347
Adding measurement 4119e82a to history. Total: 5
New Minimum: 0.01845212270699858 > 0.007701016129902769
END: th(21.544346900318843)=0.007701016129902769; dx=-0.0018215071122520757 evalInputDelta=0.08923746879260927
Fitness changed from 0.09693848492251204 to 0.007701016129902769
Iteration 4 complete. Error: 0.007701016129902769 Total: 0.2155; Orientation: 0.1752; Line Search: 0.0329
Non-optimal measurement 0.007701016129902769 < 0.007701016129902769. Total: 6
Rejected: LBFGS Orientation magnitude: 6.798e-01, gradient 2.266e-02, dot -1.000; [9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.007701016129902769, 0.01845212270699858, 0.09693848492251204, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 6.798e-01, gradient 2.266e-02, dot -1.000; [48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.007701016129902769, 0.01845212270699858, 0.09693848492251204, 1.829510740749581, 2.1492186844466117
Rejected: LBFGS Orientation magnitude: 6.798e-01, gradient 2.266e-02, dot -1.000; [b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.007701016129902769, 0.01845212270699858, 0.09693848492251204, 1.829510740749581
LBFGS Accumulation History: 3 points
Removed measurement 4119e82a to history. Total: 5
Removed measurement 59408f4a to history. Total: 4
Removed measurement c10f241 to history. Total: 3
Adding measurement 6edc7ab6 to history. Total: 3
th(0)=0.007701016129902769;dx=-5.134010753268513E-4
Adding measurement 40a9eddf to history. Total: 4
New Minimum: 0.007701016129902769 > 0.0023058672556143887
WOLF (strong): th(46.4158883361278)=0.0023058672556143887; dx=2.809311574737843E-4 evalInputDelta=0.00539514887428838
Adding measurement 180c7a9b to history. Total: 5
New Minimum: 0.0023058672556143887 > 3.947371653259078E-4
END: th(23.2079441680639)=3.947371653259078E-4; dx=-1.1623495892653354E-4 evalInputDelta=0.007306278964576861
Fitness changed from 0.007701016129902769 to 3.947371653259078E-4
Iteration 5 complete. Error: 3.947371653259078E-4 Total: 0.4653; Orientation: 0.4156; Line Search: 0.0383
Non-optimal measurement 3.947371653259078E-4 < 3.947371653259078E-4. Total: 6
Rejected: LBFGS Orientation magnitude: 1.539e-01, gradient 5.130e-03, dot -1.000; [9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.947371653259078E-4, 0.0023058672556143887, 0.007701016129902769, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 1.539e-01, gradient 5.130e-03, dot -1.000; [790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.947371653259078E-4, 0.0023058672556143887, 0.007701016129902769, 1.82951074
...skipping 28086 bytes...
+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.948803895354576E-28, 3.477812936225092E-27, 7.550028398908242E-27, 1.829510740749581, 2.1492186844466117
Rejected: LBFGS Orientation magnitude: 1.081e-13, gradient 3.604e-15, dot -1.000; [48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.948803895354576E-28, 3.477812936225092E-27, 7.550028398908242E-27, 1.829510740749581
LBFGS Accumulation History: 3 points
Removed measurement 56dfeaaa to history. Total: 5
Removed measurement 7fd4cb8a to history. Total: 4
Removed measurement 45cfd9cb to history. Total: 3
Adding measurement 2c5fdb9d to history. Total: 3
th(0)=1.948803895354576E-28;dx=-1.2992025969030507E-29
Adding measurement 47219a2d to history. Total: 4
New Minimum: 1.948803895354576E-28 > 1.2736145592542383E-28
WOLF (strong): th(54.253472222222264)=1.2736145592542383E-28; dx=1.0502860821672762E-29 evalInputDelta=6.751893361003377E-29
Adding measurement 6813a485 to history. Total: 5
New Minimum: 1.2736145592542383E-28 > 1.7674162803144445E-30
END: th(27.126736111111132)=1.7674162803144445E-30; dx=-1.2365562138248414E-30 evalInputDelta=1.9311297325514316E-28
Fitness changed from 1.948803895354576E-28 to 1.7674162803144445E-30
Iteration 18 complete. Error: 1.7674162803144445E-30 Total: 0.4960; Orientation: 0.4501; Line Search: 0.0387
Non-optimal measurement 1.7674162803144445E-30 < 1.7674162803144445E-30. Total: 6
Rejected: LBFGS Orientation magnitude: 1.030e-14, gradient 3.433e-16, dot -1.000; [6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.7674162803144445E-30, 1.2736145592542383E-28, 1.948803895354576E-28, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 1.030e-14, gradient 3.433e-16, dot -1.000; [b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.7674162803144445E-30, 1.2736145592542383E-28, 1.948803895354576E-28, 1.829510740749581, 2.1492186844466117
Rejected: LBFGS Orientation magnitude: 1.030e-14, gradient 3.433e-16, dot -1.000; [790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.7674162803144445E-30, 1.2736145592542383E-28, 1.948803895354576E-28, 1.829510740749581
LBFGS Accumulation History: 3 points
Removed measurement 6813a485 to history. Total: 5
Removed measurement 47219a2d to history. Total: 4
Removed measurement 2c5fdb9d to history. Total: 3
Adding measurement 29995719 to history. Total: 3
th(0)=1.7674162803144445E-30;dx=-1.1782775202096295E-31
Adding measurement 6dc6e6eb to history. Total: 4
New Minimum: 1.7674162803144445E-30 > 1.5756837352019162E-30
WOLF (strong): th(58.44278130511842)=1.5756837352019162E-30; dx=1.111668698102457E-31 evalInputDelta=1.9173254511252838E-31
Adding measurement 5f4d6e35 to history. Total: 5
New Minimum: 1.5756837352019162E-30 > 2.1586464793340265E-34
END: th(29.22139065255921)=2.1586464793340265E-34; dx=-5.100504469385529E-34 evalInputDelta=1.7672004156665113E-30
Fitness changed from 1.7674162803144445E-30 to 2.1586464793340265E-34
Iteration 19 complete. Error: 2.1586464793340265E-34 Total: 0.3527; Orientation: 0.3049; Line Search: 0.0397
Non-optimal measurement 2.1586464793340265E-34 < 2.1586464793340265E-34. Total: 6
Rejected: LBFGS Orientation magnitude: 1.138e-16, gradient 3.794e-18, dot -1.000; [b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.1586464793340265E-34, 1.5756837352019162E-30, 1.7674162803144445E-30, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 1.138e-16, gradient 3.794e-18, dot -1.000; [48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.1586464793340265E-34, 1.5756837352019162E-30, 1.7674162803144445E-30, 1.829510740749581, 2.1492186844466117
Rejected: LBFGS Orientation magnitude: 1.138e-16, gradient 3.794e-18, dot -1.000; [48c5a346-1974-4072-add1-f07e4a5d7d53 = 1.000/1.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 1.000/1.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 1.000/1.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 1.000/1.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.1586464793340265E-34, 1.5756837352019162E-30, 1.7674162803144445E-30, 1.829510740749581
LBFGS Accumulation History: 3 points
Removed measurement 5f4d6e35 to history. Total: 5
Removed measurement 6dc6e6eb to history. Total: 4
Removed measurement 29995719 to history. Total: 3
Adding measurement 7457e21e to history. Total: 3
th(0)=2.1586464793340265E-34;dx=-1.4390976528893514E-35
Non-optimal measurement 2.2007761968674974E-34 < 2.1586464793340265E-34. Total: 4
Armijo: th(62.955577712846996)=2.2007761968674974E-34; dx=1.4524721663920405E-35 evalInputDelta=-4.2129717533470955E-36
Adding measurement 72b9af8f to history. Total: 4
New Minimum: 2.1586464793340265E-34 > 0.0
END: th(31.477788856423498)=0.0; dx=0.0 evalInputDelta=2.1586464793340265E-34
Fitness changed from 2.1586464793340265E-34 to 0.0
Iteration 20 complete. Error: 0.0 Total: 0.4137; Orientation: 0.3750; Line Search: 0.0322
Non-optimal measurement 0.0 < 0.0. Total: 5
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [9cfca330-6e44-4107-a11c-f3c59378f61c = 0.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 0.000e+00, 48c5a346-1974-4072-add1-f07e4a5d7d53 = 0.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 0.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 2.1586464793340265E-34, 1.829510740749581, 2.1492186844466117, 2.494658133333333
Rejected: LBFGS Orientation magnitude: 0.000e+00, gradient 0.000e+00, dot NaN; [48c5a346-1974-4072-add1-f07e4a5d7d53 = 0.000e+00, b7129d56-4db7-49c7-a0ea-38e70938e304 = 0.000e+00, 790b6b51-bd12-4384-9b7f-fffcd5331566 = 0.000e+00, 9cfca330-6e44-4107-a11c-f3c59378f61c = 0.000e+00, 6855a9ed-cc8e-4b61-8c46-b30280cb12ae = 0.000e+00]
Orientation rejected. Popping history element from 0.0, 2.1586464793340265E-34, 1.829510740749581, 2.1492186844466117
LBFGS Accumulation History: 3 points
Removed measurement 72b9af8f to history. Total: 4
Removed measurement 7457e21e to history. Total: 3
Adding measurement 73c2ffe4 to history. Total: 3
th(0)=0.0;dx=0.0 (ERROR: Starting derivative negative)
Non-optimal measurement 0.0 < 0.0. Total: 4
Fitness changed from 0.0 to 0.0
Static Iteration Total: 0.2049; Orientation: 0.1807; Line Search: 0.0174
Iteration 21 failed. Error: 0.0
Previous Error: 0.0 -> 0.0
Optimization terminated 21
Final threshold in iteration 21: 0.0 (> 0.0) after 6.611s (< 30.000s)
Returns
0.0
TrainingTester.java:432 executed in 0.15 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
Plotting range=[1.0, -33.66581847607509], [20.0, 0.26233496349990526]; valueStats=DoubleSummaryStatistics{count=39, sum=5.614005, min=0.000000, average=0.143949, max=1.829511}
Plotting 20 points for GD
Plotting 2 points for CjGD
Plotting 20 points for LBFGS
Returns
TrainingTester.java:435 executed in 0.02 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
Plotting range=[0.0, -33.66581847607509], [6.338, 0.26233496349990526]; valueStats=DoubleSummaryStatistics{count=39, sum=5.614005, min=0.000000, average=0.143949, max=1.829511}
Plotting 20 points for GD
Plotting 2 points for CjGD
Plotting 20 points for LBFGS
Returns
TrainingTester.java:255 executed in 0.00 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "Converged", "value": 0.0 }, "CjGD": { "type": "Converged", "value": 0.0 }, "GD": { "type": "Converged", "value": 0.0 } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "9.095",
"gc_time": "0.404"
},
"created_on": 1586736634573,
"file_name": "trainingTest",
"report": {
"simpleName": "UR",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgTileSelectLayerTest.UR",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "Converged",
"value": 0.0
},
"CjGD": {
"type": "Converged",
"value": 0.0
},
"GD": {
"type": "Converged",
"value": 0.0
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayer/UR/trainingTest/202004131034",
"id": "469da641-47ea-42ad-b380-eaaff201be54",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "ImgTileSelectLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.cudnn.ImgTileSelectLayer",
"link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/ImgTileSelectLayer.java",
"javaDoc": ""
}
}