Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase
Using Seed 1908484623121534976
In this apply, we use a network to learn this target input, given it's pre-evaluated output:
TrainingTester.java:332 executed in 0.01 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(input_target)).flatMap(RefArrays::stream).map(x -> {
try {
return x.prettyPrint();
} finally {
x.freeRef();
}
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.032 ], [ 1.524 ], [ 1.356 ], [ -0.068 ], [ -1.616 ], [ 1.556 ] ],
[ [ 0.048 ], [ 1.912 ], [ -0.804 ], [ 0.08 ], [ 0.788 ], [ -1.688 ] ],
[ [ 0.028 ], [ -0.768 ], [ 0.3 ], [ -0.128 ], [ -0.852 ], [ 1.512 ] ],
[ [ 0.636 ], [ 0.496 ], [ -0.712 ], [ -0.176 ], [ 1.108 ], [ 1.048 ] ],
[ [ -1.028 ], [ 0.392 ], [ -0.608 ], [ 1.556 ], [ 0.7 ], [ -1.72 ] ],
[ [ 1.208 ], [ 1.64 ], [ 1.764 ], [ 1.612 ], [ -0.384 ], [ 0.092 ] ]
]
[
[ [ -0.852 ], [ 0.7 ], [ -1.688 ], [ 0.3 ], [ 1.912 ], [ -0.068 ] ],
[ [ -0.128 ], [ 1.556 ], [ -0.768 ], [ 1.524 ], [ 1.612 ], [ 0.788 ] ],
[ [ -1.72 ], [ 1.108 ], [ 1.356 ], [ 1.556 ], [ 1.032 ], [ 1.208 ] ],
[ [ -0.384 ], [ -0.176 ], [ -0.608 ], [ -1.028 ], [ 1.64 ], [ 0.392 ] ],
[ [ 1.048 ], [ 0.496 ], [ 0.028 ], [ 0.092 ], [ 0.08 ], [ 1.512 ] ],
[ [ -0.712 ], [ -1.616 ], [ 0.636 ], [ 1.764 ], [ 0.048 ], [ -0.804 ] ]
]
[
[ [ 1.912 ], [ -1.028 ], [ -0.712 ], [ -0.176 ], [ 0.496 ], [ -0.804 ] ],
[ [ 1.356 ], [ 1.764 ], [ 0.048 ], [ 1.208 ], [ 1.512 ], [ 1.032 ] ],
[ [ 1.108 ], [ 1.612 ], [ -0.384 ], [ 0.392 ], [ 1.64 ], [ 0.3 ] ],
[ [ -0.852 ], [ -0.768 ], [ -1.72 ], [ -0.608 ], [ 1.048 ], [ 0.7 ] ],
[ [ -1.688 ], [ 0.028 ], [ 0.092 ], [ 0.636 ], [ 1.556 ], [ 1.524 ] ],
[ [ 1.556 ], [ -1.616 ], [ 0.788 ], [ -0.128 ], [ 0.08 ], [ -0.068 ] ]
]
[
[ [ 1.912 ], [ 1.208 ], [ 1.764 ], [ 0.788 ], [ 0.048 ], [ -0.068 ] ],
[ [ 0.496 ], [ -1.028 ], [ 1.032 ], [ 0.636 ], [ 1.556 ], [ 1.556 ] ],
[ [ 0.3 ], [ -0.384 ], [ -0.852 ], [ 0.092 ], [ 0.7 ], [ -1.72 ] ],
[ [ 1.524 ], [ -0.608 ], [ 1.048 ], [ -1.616 ], [ -0.804 ], [ 1.612 ] ],
[ [ -1.688 ], [ 1.108 ], [ 0.08 ], [ 1.356 ], [ -0.176 ], [ 0.028 ] ],
[ [ -0.768 ], [ 1.512 ], [ 0.392 ], [ 1.64 ], [ -0.712 ], [ -0.128 ] ]
]
[
[ [ 0.08 ], [ -0.608 ], [ 1.764 ], [ 0.048 ], [ 1.108 ], [ 1.208 ] ],
[ [ 1.912 ], [ 0.392 ], [ 1.032 ], [ 0.3 ], [ 0.7 ], [ 1.512 ] ],
[ [ -1.72 ], [ -1.688 ], [ 1.556 ], [ -0.176 ], [ 1.556 ], [ 1.612 ] ],
[ [ -0.804 ], [ 0.028 ], [ 0.092 ], [ 1.64 ], [ -1.616 ], [ 1.048 ] ],
[ [ 0.636 ], [ -0.068 ], [ 1.356 ], [ -1.028 ], [ -0.384 ], [ 0.788 ] ],
[ [ -0.852 ], [ -0.768 ], [ 1.524 ], [ -0.712 ], [ 0.496 ], [ -0.128 ] ]
]
First, we train using basic gradient descent method apply weak line search conditions.
TrainingTester.java:480 executed in 0.03 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 865915426117
Set StochasticSamplingSubnetLayer to random seed 865915426117
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 0: 0.0 (> 0.0) after 0.021s (< 30.000s)
Returns
0.0
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.512 ], [ -0.128 ], [ 0.08 ], [ 1.764 ], [ 1.912 ], [ 0.048 ] ],
[ [ 1.048 ], [ 1.64 ], [ 1.556 ], [ 0.788 ], [ 1.208 ], [ 0.7 ] ],
[ [ -0.176 ], [ -1.72 ], [ -0.852 ], [ 1.612 ], [ -1.028 ], [ 0.092 ] ],
[ [ -0.768 ], [ 1.108 ], [ 0.496 ], [ 0.028 ], [ 0.392 ], [ -0.384 ] ],
[ [ 1.556 ], [ -1.688 ], [ 0.636 ], [ -1.616 ], [ 0.3 ], [ -0.608 ] ],
[ [ 1.356 ], [ -0.068 ], [ 1.524 ], [ -0.712 ], [ -0.804 ], [ 1.032 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ -0.0 ]
First, we use a conjugate gradient descent method, which converges the fastest for purely linear functions.
TrainingTester.java:452 executed in 0.02 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new QuadraticSearch());
iterativeTrainer.setOrientation(new GradientDescent());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 865951517911
Set StochasticSamplingSubnetLayer to random seed 865951517911
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 0: 0.0 (> 0.0) after 0.020s (< 30.000s)
Returns
0.0
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.512 ], [ -0.128 ], [ 0.08 ], [ 1.764 ], [ 1.912 ], [ 0.048 ] ],
[ [ 1.048 ], [ 1.64 ], [ 1.556 ], [ 0.788 ], [ 1.208 ], [ 0.7 ] ],
[ [ -0.176 ], [ -1.72 ], [ -0.852 ], [ 1.612 ], [ -1.028 ], [ 0.092 ] ],
[ [ -0.768 ], [ 1.108 ], [ 0.496 ], [ 0.028 ], [ 0.392 ], [ -0.384 ] ],
[ [ 1.556 ], [ -1.688 ], [ 0.636 ], [ -1.616 ], [ 0.3 ], [ -0.608 ] ],
[ [ 1.356 ], [ -0.068 ], [ 1.524 ], [ -0.712 ], [ -0.804 ], [ 1.032 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ -0.0 ]
Next, we apply the same optimization using L-BFGS, which is nearly ideal for purely second-order or quadratic functions.
TrainingTester.java:509 executed in 0.01 seconds (0.000 gc):
IterativeTrainer iterativeTrainer = new IterativeTrainer(trainable.addRef());
try {
iterativeTrainer.setLineSearchFactory(label -> new ArmijoWolfeSearch());
iterativeTrainer.setOrientation(new LBFGS());
iterativeTrainer.setMonitor(TrainingTester.getMonitor(history));
iterativeTrainer.setTimeout(30, TimeUnit.SECONDS);
iterativeTrainer.setIterationsPerSample(100);
iterativeTrainer.setMaxIterations(250);
iterativeTrainer.setTerminateThreshold(0);
return iterativeTrainer.run();
} finally {
iterativeTrainer.freeRef();
}
Reset training subject: 865980895225
Set StochasticSamplingSubnetLayer to random seed 865980895225
Set StochasticSamplingSubnetLayer to random null seed
Final threshold in iteration 0: 0.0 (> 0.0) after 0.012s (< 30.000s)
Returns
0.0
This training apply resulted in the following configuration:
TrainingTester.java:610 executed in 0.00 seconds (0.000 gc):
RefList<double[]> state = network.state();
assert state != null;
String description = state.stream().map(RefArrays::toString).reduce((a, b) -> a + "\n" + b)
.orElse("");
state.freeRef();
return description;
Returns
TrainingTester.java:622 executed in 0.00 seconds (0.000 gc):
return RefArrays.stream(RefUtil.addRef(data)).flatMap(x -> {
return RefArrays.stream(x);
}).limit(1).map(x -> {
String temp_18_0015 = x.prettyPrint();
x.freeRef();
return temp_18_0015;
}).reduce((a, b) -> a + "\n" + b).orElse("");
Returns
[
[ [ 1.512 ], [ -0.128 ], [ 0.08 ], [ 1.764 ], [ 1.912 ], [ 0.048 ] ],
[ [ 1.048 ], [ 1.64 ], [ 1.556 ], [ 0.788 ], [ 1.208 ], [ 0.7 ] ],
[ [ -0.176 ], [ -1.72 ], [ -0.852 ], [ 1.612 ], [ -1.028 ], [ 0.092 ] ],
[ [ -0.768 ], [ 1.108 ], [ 0.496 ], [ 0.028 ], [ 0.392 ], [ -0.384 ] ],
[ [ 1.556 ], [ -1.688 ], [ 0.636 ], [ -1.616 ], [ 0.3 ], [ -0.608 ] ],
[ [ 1.356 ], [ -0.068 ], [ 1.524 ], [ -0.712 ], [ -0.804 ], [ 1.032 ] ]
]
To produce the following output:
TrainingTester.java:633 executed in 0.00 seconds (0.000 gc):
Result[] array = ConstantResult.batchResultArray(pop(RefUtil.addRef(data)));
@Nullable
Result eval = layer.eval(array);
assert eval != null;
TensorList tensorList = Result.getData(eval);
String temp_18_0016 = tensorList.stream().limit(1).map(x -> {
String temp_18_0017 = x.prettyPrint();
x.freeRef();
return temp_18_0017;
}).reduce((a, b) -> a + "\n" + b).orElse("");
tensorList.freeRef();
return temp_18_0016;
Returns
[ -0.0 ]
TrainingTester.java:432 executed in 0.00 seconds (0.000 gc):
return TestUtil.compare(title + " vs Iteration", runs);
No Data
TrainingTester.java:435 executed in 0.00 seconds (0.000 gc):
return TestUtil.compareTime(title + " vs Time", runs);
No Data
TrainingTester.java:255 executed in 0.03 seconds (0.000 gc):
return grid(inputLearning, modelLearning, completeLearning);
Returns
TrainingTester.java:258 executed in 0.00 seconds (0.000 gc):
return new ComponentResult(null == inputLearning ? null : inputLearning.value,
null == modelLearning ? null : modelLearning.value, null == completeLearning ? null : completeLearning.value);
Returns
{"input":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }, "model":null, "complete":null}
LayerTests.java:425 executed in 0.00 seconds (0.000 gc):
throwException(exceptions.addRef());
| details | result |
|---|---|
| {"input":{ "LBFGS": { "type": "NonConverged", "value": NaN }, "CjGD": { "type": "NonConverged", "value": NaN }, "GD": { "type": "NonConverged", "value": NaN } }, "model":null, "complete":null} | OK |
{
"result": "OK",
"performance": {
"execution_time": "0.390",
"gc_time": "0.139"
},
"created_on": 1586735453712,
"file_name": "trainingTest",
"report": {
"simpleName": "Basic",
"canonicalName": "com.simiacryptus.mindseye.layers.java.StochasticSamplingSubnetLayerTest.Basic",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/StochasticSamplingSubnetLayerTest.java",
"javaDoc": ""
},
"training_analysis": {
"input": {
"LBFGS": {
"type": "NonConverged",
"value": "NaN"
},
"CjGD": {
"type": "NonConverged",
"value": "NaN"
},
"GD": {
"type": "NonConverged",
"value": "NaN"
}
}
},
"archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/StochasticSamplingSubnetLayer/Basic/trainingTest/202004125053",
"id": "251ca6e1-001b-4b94-95f8-d7bc24297e07",
"report_type": "Components",
"display_name": "Comparative Training",
"target": {
"simpleName": "StochasticSamplingSubnetLayer",
"canonicalName": "com.simiacryptus.mindseye.layers.java.StochasticSamplingSubnetLayer",
"link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/StochasticSamplingSubnetLayer.java",
"javaDoc": ""
}
}