1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 10.01 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
java.lang.reflect.InvocationTargetException 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:659)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.LayerTests.renderGraph(LayerTests.java:203)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:415)
at com.simiacryptus.mindseye.test.LayerTestBase.derivativeTest(LayerTestBase.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(Unknown Source)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(Unknown Source)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambda$null$8bb7732f$1(MacroTestRunner.java:177)
at com.simiacryptus.aws.TendrilControl.lambda$null$0(TendrilControl.java:86)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy44.get(Unknown Source)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
... 71 more
Caused by: guru.nidi.graphviz.engine.GraphvizException: Engine took too long to respond, try setting a higher timout
at guru.nidi.graphviz.engine.GraphvizServerEngine.execute(GraphvizServerEngine.java:49)
at guru.nidi.graphviz.engine.Graphviz.execute(Graphviz.java:246)
at guru.nidi.graphviz.engine.Renderer.execute(Renderer.java:134)
at guru.nidi.graphviz.engine.Renderer.toImage(Renderer.java:105)
at com.simiacryptus.mindseye.test.unit.LayerTests.lambda$renderGraph$12c65ef7$1(LayerTests.java:205)
... 77 more

Using Seed 4681532977821545472

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [ 0.08, 0.7, -0.128 ]
Inputs Statistics: {meanExponent=-0.7148673344486438, negative=1, min=-0.128, max=0.7, mean=0.2173333333333333, count=3, sum=0.6519999999999999, positive=2, stdDev=0.3517018939701949, zeros=0}
Output: [ 0.649751613290642 ]
Outputs Statistics: {meanExponent=-0.1872526335008956, negative=0, min=0.649751613290642, max=0.649751613290642, mean=0.649751613290642, count=1, sum=0.649751613290642, positive=1, stdDev=0.0, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.02 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [ 0.08, 0.7, -0.128 ]
Value Statistics: {meanExponent=-0.7148673344486438, negative=1, min=-0.128, max=0.7, mean=0.2173333333333333, count=3, sum=0.6519999999999999, positive=2, stdDev=0.3517018939701949, zeros=0}
Implemented Feedback: [ [ 0.26773394776908155 ], [ 0.03277274069742764 ], [ 0.3465596430447332 ] ]
Implemented Statistics: {meanExponent=-0.8390019362709854, negative=0, min=0.03277274069742764, max=0.3465596430447332, mean=0.21568877717041413, count=3, sum=0.6470663315112424, positive=3, stdDev=0.13328435731713523, zeros=0}
Measured Feedback: [ [ 0.26772746726799745 ], [ 0.03276767163384875 ], [ 0.34657744669597257 ] ]
Measured Statistics: {meanExponent=-0.8390203965658478, negative=0, min=0.03276767163384875, max=0.34657744669597257, mean=0.21569086186593958, count=3, sum=0.6470725855978188, positive=3, stdDev=0.13329166004783718, zeros=0}
Feedback Error: [ [ -6.480501084094037E-6 ], [ -5.0690635788885885E-6 ], [ 1.780365123937866E-5 ] ]
Error Statistics: {meanExponent=-5.077651531934954, negative=2, min=-6.480501084094037E-6, max=1.780365123937866E-5, mean=2.0846955254653445E-6, count=3, sum=6.254086576396034E-6, positive=1, stdDev=1.1129906124484344E-5, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 3,
        "sum" : 2.9353215902361285E-5,
        "min" : 5.0690635788885885E-6,
        "max" : 1.780365123937866E-5,
        "sumOfSquare" : 3.8466229732118805E-10,
        "standardDeviation" : 5.69966478695535E-6,
        "average" : 9.784405300787094E-6
      },
      "relativeTol" : {
        "count" : 3,
        "sum" : 1.1513082606071616E-4,
        "min" : 1.210264811264745E-5,
        "max" : 7.734256466478763E-5,
        "sumOfSquare" : 6.78809713000369E-9,
        "standardDeviation" : 2.8105326265848603E-5,
        "average" : 3.837694202023872E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.03 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Learning Gradient for weight setByCoord 0
Weights: [ -1.0, 0.0 ]
Implemented Gradient: [ [ -0.026850255099728045 ], [ -0.5684545332378723 ] ]
Implemented Statistics: {meanExponent=-0.9081779247987342, negative=2, min=-0.5684545332378723, max=-0.026850255099728045, mean=-0.2976523941688002, count=2, sum=-0.5953047883376004, positive=0, stdDev=0.27080213906907213, zeros=0}
Measured Gradient: [ [ -0.026849533584227814 ], [ -0.5681313278416145 ] ]
Measured Statistics: {meanExponent=-0.9083072582349576, negative=2, min=-0.5681313278416145, max=-0.026849533584227814, mean=-0.29749043071292114, count=2, sum=-0.5949808614258423, positive=0, stdDev=0.2706408971286933, zeros=0}
Gradient Error: [ [ 7.215155002314455E-7 ], [ 3.2320539625785294E-4 ] ]
Error Statistics: {meanExponent=-4.816137865701167, negative=0, min=7.215155002314455E-7, max=3.2320539625785294E-4, mean=1.619634558790422E-4, count=2, sum=3.239269117580844E-4, positive=2, stdDev=1.6124194037881075E-4, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 5,
        "sum" : 3.5328012766044567E-4,
        "min" : 7.215155002314455E-7,
        "max" : 3.2320539625785294E-4,
        "sumOfSquare" : 1.04846911052134E-7,
        "standardDeviation" : 1.2640058649561548E-4,
        "average" : 7.065602553208913E-5
      },
      "relativeTol" : {
        "count" : 5,
        "sum" : 4.129320606684237E-4,
        "min" : 1.210264811264745E-5,
        "max" : 2.8436513890018353E-4,
        "sumOfSquare" : 8.783215801958611E-8,
        "standardDeviation" : 1.0366251074907645E-4,
        "average" : 8.258641213368474E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 7.0656e-05 +- 1.2640e-04 [7.2152e-07 - 3.2321e-04] (5#)
relativeTol: 8.2586e-05 +- 1.0366e-04 [1.2103e-05 - 2.8437e-04] (5#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.03 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=7.0656e-05 +- 1.2640e-04 [7.2152e-07 - 3.2321e-04] (5#), relativeTol=8.2586e-05 +- 1.0366e-04 [1.2103e-05 - 2.8437e-04] (5#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "10.254",
      "gc_time": "0.131"
    },
    "created_on": 1586738289143,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Normal",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.SignReducerLayerTest.Normal",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/SignReducerLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/SignReducerLayer/Normal/derivativeTest/202004133809",
    "id": "0751aef3-9fb5-4763-ac3a-da522c94d3f6",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "SignReducerLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.SignReducerLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/SignReducerLayer.java",
      "javaDoc": ""
    }
  }