1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
  4. SubTests: PipelineNetwork
  5. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging 
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine8889496147707385956/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 6016587263958315008

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08 ], [ -0.608 ], [ 1.208 ], [ -0.852 ] ],
[ [ 0.7 ], [ 1.764 ], [ -1.72 ], [ 1.912 ] ],
[ [ -0.128 ], [ 0.048 ], [ -1.028 ], [ -1.688 ] ],
[ [ 0.496 ], [ 1.524 ], [ -0.384 ], [ -0.804 ] ]
]
Inputs Statistics: {meanExponent=-0.20600101677773192, negative=8, min=-1.72, max=1.912, mean=0.0325, count=16, sum=0.52, positive=8, stdDev=1.1193595266937248, zeros=0}
Output: [
[ [ 0.16 ], [ -1.216 ], [ 2.416 ], [ -1.704 ] ],
[ [ 1.4 ], [ 3.528 ], [ -3.44 ], [ 3.824 ] ],
[ [ -0.256 ], [ 0.096 ], [ -2.056 ], [ -3.376 ] ],
[ [ 0.992 ], [ 3.048 ], [ -0.768 ], [ -1.608 ] ]
]
Outputs Statistics: {meanExponent=0.09502897888624931, negative=8, min=-3.44, max=3.824, mean=0.065, count=16, sum=1.04, positive=8, stdDev=2.2387190533874497, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.17 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08 ], [ -0.608 ], [ 1.208 ], [ -0.852 ] ],
[ [ 0.7 ], [ 1.764 ], [ -1.72 ], [ 1.912 ] ],
[ [ -0.128 ], [ 0.048 ], [ -1.028 ], [ -1.688 ] ],
[ [ 0.496 ], [ 1.524 ], [ -0.384 ], [ -0.804 ] ]
]
Value Statistics: {meanExponent=-0.20600101677773192, negative=8, min=-1.72, max=1.912, mean=0.0325, count=16, sum=0.52, positive=8, stdDev=1.1193595266937248, zeros=0}
Implemented Feedback: [ [ 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, ... ], ... ]
Implemented Statistics: {meanExponent=0.0, negative=0, min=0.0, max=1.0, mean=0.0625, count=256, sum=16.0, positive=16, stdDev=0.24206145913796356, zeros=240}
Measured: [ [ 2.0000000000000573, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 1.9999999999997797, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 1.9999999999997797, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 1.9999999999997797, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.9999999999997797, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.9999999999997797, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0000000000000573, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.9999999999997797, ... ], ... ]
Measured Statistics: {meanExponent=0.30102999566394084, negative=0, min=0.0, max=2.0000000000000573, mean=0.1249999999999884, count=256, sum=31.99999999999703, positive=16, stdDev=0.48412291827588216, zeros=240}
Feedback Error: [ [ 1.0000000000000573, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.9999999999997797, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.9999999999997797, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.9999999999997797, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.9999999999997797, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999997797, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0000000000000573, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999997797, ... ], ... ]
Error Statistics: {meanExponent=-8.059366801720026E-14, negative=0, min=0.0, max=1.0000000000000573, mean=0.0624999999999884, count=256, sum=15.99999999999703, positive=16, stdDev=0.24206145913791863, zeros=240}
java.lang.reflect.InvocationTargetException 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:659)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:117)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:44)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
at com.simiacryptus.mindseye.test.LayerTestBase.derivativeTest(LayerTestBase.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambda$null$8bb7732f$1(MacroTestRunner.java:177)
at com.simiacryptus.aws.TendrilControl.lambda$null$0(TendrilControl.java:86)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy44.get(Unknown Source)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
... 73 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy52.apply(Unknown Source)
at com.simiacryptus.ref.wrappers.RefIntStream.lambda$mapToObj$10(RefIntStream.java:251)
at java.util.stream.IntPipeline$4$1.accept(IntPipeline.java:250)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:546)
at com.simiacryptus.ref.wrappers.StreamWrapper.reduce(StreamWrapper.java:211)
at com.simiacryptus.ref.wrappers.RefStream.reduce(RefStream.java:477)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.testFeedback(SingleDerivativeTester.java:339)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$test$fbe4b857$1(SingleDerivativeTester.java:118)
... 79 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 93 more
Caused by: java.lang.AssertionError: ToleranceStatistics{absoluteTol=6.2500e-02 +- 2.4206e-01 [0.0000e+00 - 1.0000e+00] (256#), relativeTol=3.3333e-01 +- NaN [3.3333e-01 - 3.3333e-01] (16#)}
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$testFeedback$11(SingleDerivativeTester.java:296)
... 97 more

SubTests: PipelineNetwork

[[4, 4, 1], [4, 4, 1]]

LayerTests.java:241 executed in 0.00 seconds (0.000 gc):

    return new GsonBuilder().setPrettyPrinting().create().toJson(
        subLayer.getJson(new HashMap<>(), SerialPrecision.Double)
    );

Returns

    {
      "class": "com.simiacryptus.mindseye.layers.cudnn.SumInputsLayer",
      "id": "5858d365-f127-48f5-9dd7-0878d64a451e",
      "isFrozen": false,
      "name": "SumInputsLayer",
      "precision": "Double",
      "parallel": true
    }

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 1.108 ], [ -1.616 ], [ 1.032 ], [ 1.556 ] ],
[ [ 0.028 ], [ 1.356 ], [ 0.3 ], [ 0.788 ] ],
[ [ -0.712 ], [ 1.512 ], [ 0.636 ], [ -0.768 ] ],
[ [ 1.048 ], [ 1.556 ], [ -0.176 ], [ -0.068 ] ]
],
[
[ [ 1.612 ], [ -0.556 ], [ 1.552 ], [ -1.572 ] ],
[ [ 1.64 ], [ -1.476 ], [ 1.876 ], [ -1.516 ] ],
[ [ 0.392 ], [ 1.704 ], [ -0.408 ], [ -0.636 ] ],
[ [ 0.092 ], [ -1.228 ], [ -0.384 ], [ -1.492 ] ]
]
Inputs Statistics: {meanExponent=-0.22356386398987144, negative=5, min=-1.616, max=1.556, mean=0.47375000000000006, count=16, sum=7.580000000000001, positive=11, stdDev=0.92287482222672, zeros=0},
{meanExponent=-0.04827465585032461, negative=9, min=-1.572, max=1.876, mean=-0.025000000000000078, count=16, sum=-0.40000000000000124, positive=7, stdDev=1.2743033390837522, zeros=0}
Output: [
[ [ 2.72 ], [ -2.172 ], [ 2.584 ], [ -0.016000000000000014 ] ],
[ [ 1.668 ], [ -0.11999999999999988 ], [ 2.1759999999999997 ], [ -0.728 ] ],
[ [ -0.31999999999999995 ], [ 3.216 ], [ 0.22800000000000004 ], [ -1.404 ] ],
[ [ 1.1400000000000001 ], [ 0.32800000000000007 ], [ -0.56 ], [ -1.56 ] ]
]
Outputs Statistics: {meanExponent=-0.12994574542388265, negative=8, min=-2.172, max=3.216, mean=0.44875, count=16, sum=7.18, positive=8, stdDev=1.5861804555283108, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.18 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 1.108 ], [ -1.616 ], [ 1.032 ], [ 1.556 ] ],
[ [ 0.028 ], [ 1.356 ], [ 0.3 ], [ 0.788 ] ],
[ [ -0.712 ], [ 1.512 ], [ 0.636 ], [ -0.768 ] ],
[ [ 1.048 ], [ 1.556 ], [ -0.176 ], [ -0.068 ] ]
]
Value Statistics: {meanExponent=-0.22356386398987144, negative=5, min=-1.616, max=1.556, mean=0.47375000000000006, count=16, sum=7.580000000000001, positive=11, stdDev=0.92287482222672, zeros=0}
Implemented Feedback: [ [ 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, ... ], ... ]
Implemented Statistics: {meanExponent=0.0, negative=0, min=0.0, max=1.0, mean=0.0625, count=256, sum=16.0, positive=16, stdDev=0.24206145913796356, zeros=240}
Measured Feedback: [ [ 1.0000000000021103, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.0000000000021103, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999976694, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999998899, ... ], ... ]
Measured Statistics: {meanExponent=7.271029097772705E-14, negative=0, min=0.0, max=1.0000000000021103, mean=0.06250000000001046, count=256, sum=16.00000000000268, positive=16, stdDev=0.2420614591380041, zeros=240}
Feedback Error: [ [ 2.1103119252074976E-12, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 2.1103119252074976E-12, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -2.3305801732931286E-12, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.1013412404281553E-13, ... ], ... ]
Error Statistics: {meanExponent=-12.47177914205742, negative=12, min=-2.3305801732931286E-12, max=2.1103119252074976E-12, mean=1.04638520070921E-14, count=256, sum=2.6787461138155777E-12, positive=4, stdDev=3.352362671082306E-13, zeros=240}
Feedback for input 1
Inputs Values: [
[ [ 1.612 ], [ -0.556 ], [ 1.552 ], [ -1.572 ] ],
[ [ 1.64 ], [ -1.476 ], [ 1.876 ], [ -1.516 ] ],
[ [ 0.392 ], [ 1.704 ], [ -0.408 ], [ -0.636 ] ],
[ [ 0.092 ], [ -1.228 ], [ -0.384 ], [ -1.492 ] ]
]
Value Statistics: {meanExponent=-0.04827465585032461, negative=9, min=-1.572, max=1.876, mean=-0.025000000000000078, count=16, sum=-0.40000000000000124, positive=7, stdDev=1.2743033390837522, zeros=0}
Implemented Feedback: [ [ 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, ... ], ... ]
Implemented Statistics: {meanExponent=0.0, negative=0, min=0.0, max=1.0, mean=0.0625, count=256, sum=16.0, positive=16, stdDev=0.24206145913796356, zeros=240}
Measured Feedback: [ [ 1.0000000000021103, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 1.0000000000021103, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999998899, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999976694, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9999999999998899, ... ], ... ]
Measured Statistics: {meanExponent=7.271029097772705E-14, negative=0, min=0.0, max=1.0000000000021103, mean=0.06250000000001046, count=256, sum=16.00000000000268, positive=16, stdDev=0.2420614591380041, zeros=240}
Feedback Error: [ [ 2.1103119252074976E-12, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 2.1103119252074976E-12, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, -1.1013412404281553E-13, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -2.3305801732931286E-12, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.1013412404281553E-13, ... ], ... ]
Error Statistics: {meanExponent=-12.47177914205742, negative=12, min=-2.3305801732931286E-12, max=2.1103119252074976E-12, mean=1.04638520070921E-14, count=256, sum=2.6787461138155777E-12, positive=4, stdDev=3.352362671082306E-13, zeros=240}

Returns

    {
      "absoluteTol" : {
        "count" : 512,
        "sum" : 2.8407498575688805E-11,
        "min" : 0.0,
        "max" : 2.3305801732931286E-12,
        "sumOfSquare" : 5.759633765554548E-23,
        "standardDeviation" : 3.307785358665322E-13,
        "average" : 5.54833956556422E-14
      },
      "relativeTol" : {
        "count" : 32,
        "sum" : 1.4203749287840989E-11,
        "min" : 5.50670620214108E-14,
        "max" : 1.1652900866479222E-12,
        "sumOfSquare" : 1.439908441388024E-23,
        "standardDeviation" : 5.029446565489069E-13,
        "average" : 4.438671652450309E-13
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 512,
        "sum" : 2.8407498575688805E-11,
        "min" : 0.0,
        "max" : 2.3305801732931286E-12,
        "sumOfSquare" : 5.759633765554548E-23,
        "standardDeviation" : 3.307785358665322E-13,
        "average" : 5.54833956556422E-14
      },
      "relativeTol" : {
        "count" : 32,
        "sum" : 1.4203749287840989E-11,
        "min" : 5.50670620214108E-14,
        "max" : 1.1652900866479222E-12,
        "sumOfSquare" : 1.439908441388024E-23,
        "standardDeviation" : 5.029446565489069E-13,
        "average" : 4.438671652450309E-13
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 5.5483e-14 +- 3.3078e-13 [0.0000e+00 - 2.3306e-12] (512#)
relativeTol: 4.4387e-13 +- 5.0294e-13 [5.5067e-14 - 1.1653e-12] (32#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.06 seconds (0.059 gc):

    throwException(exceptions.addRef());
Logging 
LayerBase: PipelineNetwork
Error
null 
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy50.run(Unknown Source)
at com.simiacryptus.notebook.NotebookOutput.lambda$run$6e9b516b$1(NotebookOutput.java:105)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.run(NotebookOutput.java:104)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:425)
at com.simiacryptus.mindseye.test.LayerTestBase.derivativeTest(LayerTestBase.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:141)
at org.junit.platform.runner.JUnitPlatform.run(JUnitPlatform.java:139)
at com.simiacryptus.util.test.MacroTestRunner.runTest(MacroTestRunner.java:94)
at com.simiacryptus.util.test.MacroTestRunner.lambda$null$8bb7732f$1(MacroTestRunner.java:177)
at com.simiacryptus.aws.TendrilControl.lambda$null$0(TendrilControl.java:86)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 73 more
Caused by: com.simiacryptus.mindseye.test.unit.TestError: Error in SingleDerivativeTester{probeSize=1.0E-4, tolerance=0.001, testFeedback=true, testLearning=true, verbose=true, verify=true} apply PipelineNetwork
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:377)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:418)
... 66 more
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.simiacryptus.util.Util.throwException(Util.java:502)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:659)
at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:102)
at com.simiacryptus.notebook.MarkdownNotebookOutput.eval(MarkdownNotebookOutput.java:649)
at com.simiacryptus.notebook.NotebookOutput.eval(NotebookOutput.java:125)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:117)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:44)
at com.simiacryptus.mindseye.test.unit.LayerTests.run(LayerTests.java:369)
... 67 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
at com.sun.proxy.$Proxy44.get(Unknown Source)
at com.simiacryptus.notebook.MarkdownNotebookOutput.lambda$eval$dc58be99$1(MarkdownNotebookOutput.java:657)
... 73 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy52.apply(Unknown Source)
at com.simiacryptus.ref.wrappers.RefIntStream.lambda$mapToObj$10(RefIntStream.java:251)
at java.util.stream.IntPipeline$4$1.accept(IntPipeline.java:250)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:546)
at com.simiacryptus.ref.wrappers.StreamWrapper.reduce(StreamWrapper.java:211)
at com.simiacryptus.ref.wrappers.RefStream.reduce(RefStream.java:477)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.testFeedback(SingleDerivativeTester.java:339)
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$test$fbe4b857$1(SingleDerivativeTester.java:118)
... 79 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simiacryptus.ref.lang.RefUtil$RefWrapperHandler.invoke(RefUtil.java:238)
... 93 more
Caused by: java.lang.AssertionError: ToleranceStatistics{absoluteTol=6.2500e-02 +- 2.4206e-01 [0.0000e+00 - 1.0000e+00] (256#), relativeTol=3.3333e-01 +- NaN [3.3333e-01 - 3.3333e-01] (16#)}
at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$testFeedback$11(SingleDerivativeTester.java:296)
... 97 more



  {
    "result": "InvocationTargetException / InvocationTargetException / InvocationTargetException / AssertionError",
    "performance": {
      "execution_time": "0.827",
      "gc_time": "0.274"
    },
    "created_on": 1586742192120,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "OnePlusOne",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.SumInputsLayerTest.OnePlusOne",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/SumInputsLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/SumInputsLayer/OnePlusOne/derivativeTest/202004134312",
    "id": "3032104e-b211-4d66-ae9e-567f1ade5d5f",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "SumInputsLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.SumInputsLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/SumInputsLayer.java",
      "javaDoc": ""
    }
  }