1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging 
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine479560405527787187/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 6298290218428493824

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556 ], [ 0.496, -0.852, 1.356, -0.768, -1.228 ], [ 0.048, -0.804, 1.032, 1.64, -0.408 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476 ], [ -0.608, 1.912, 1.512, -0.068, 1.552 ], [ 1.524, 1.108, 0.3, 0.392, -0.384 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704 ], [ 1.764, -1.688, 1.556, 1.612, 1.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572 ] ]
]
Inputs Statistics: {meanExponent=-0.1734079494659544, negative=19, min=-1.72, max=1.912, mean=0.2520888888888889, count=45, sum=11.344000000000001, positive=26, stdDev=1.110411491739731, zeros=0}
Output: [
[ [ 7.421712000000001, 1.937392000000001, -0.53672 ], [ -2.366048000000001, 4.217312, 0.6795840000000009 ], [ -2.5897600000000005, -7.569952000000001, 2.5853920000000006 ] ],
[ [ 12.077791999999999, 1.8255839999999983, 1.137712 ], [ 5.547647999999999, -13.268016000000001, 11.932640000000001 ], [ -4.483104000000001, -4.839376, 4.516704000000001 ] ],
[ [ 6.804031999999999, 5.91816, -2.51696 ], [ 4.150128000000001, 6.263199999999999, -2.9276159999999987 ], [ -7.0639520000000005, 2.18432, 3.4312639999999996 ] ]
]
Outputs Statistics: {meanExponent=0.5692197471266732, negative=10, min=-13.268016000000001, max=12.077791999999999, mean=1.2766322962962962, count=27, sum=34.469072, positive=17, stdDev=5.749087444178887, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 1.85 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556 ], [ 0.496, -0.852, 1.356, -0.768, -1.228 ], [ 0.048, -0.804, 1.032, 1.64, -0.408 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476 ], [ -0.608, 1.912, 1.512, -0.068, 1.552 ], [ 1.524, 1.108, 0.3, 0.392, -0.384 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704 ], [ 1.764, -1.688, 1.556, 1.612, 1.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572 ] ]
]
Value Statistics: {meanExponent=-0.1734079494659544, negative=19, min=-1.72, max=1.912, mean=0.2520888888888889, count=45, sum=11.344000000000001, positive=26, stdDev=1.110411491739731, zeros=0}
Implemented Feedback: [ [ 1.324, -1.248, 0.0, -1.724, -1.156, 0.0, 0.0, 0.0, ... ], [ 1.352, 1.324, -1.248, -1.664, -1.724, -1.156, 0.0, 0.0, ... ], [ 0.0, 1.352, 1.324, 0.0, -1.664, -1.724, 0.0, 0.0, ... ], [ -0.888, -1.484, 0.0, 1.324, -1.248, 0.0, -1.724, -1.156, ... ], [ -1.256, -0.888, -1.484, 1.352, 1.324, -1.248, -1.664, -1.724, ... ], [ 0.0, -1.256, -0.888, 0.0, 1.352, 1.324, 0.0, -1.664, ... ], [ 0.0, 0.0, 0.0, -0.888, -1.484, 0.0, 1.324, -1.248, ... ], [ 0.0, 0.0, 0.0, -1.256, -0.888, -1.484, 1.352, 1.324, ... ], ... ]
Implemented Statistics: {meanExponent=-0.10972995230371348, negative=362, min=-2.0, max=1.98, mean=0.016148148148148102, count=1215, sum=19.619999999999944, positive=373, stdDev=0.9236712816193547, zeros=480}
Measured Feedback: [ [ 1.324000000000325, -1.24800000000036, 0.0, -1.7240000000029454, -1.1559999999999349, 0.0, 0.0, 0.0, ... ], [ 1.3519999999900278, 1.324000000000325, -1.24800000000036, -1.6639999999767952, -1.7239999999940636, -1.1559999999999349, 0.0, 0.0, ... ], [ 0.0, 1.3520000000077914, 1.324000000000325, 0.0, -1.6640000000034405, -1.7240000000029454, 0.0, 0.0, ... ], [ -0.8880000000033306, -1.4840000000049258, 0.0, 1.3240000000180885, -1.24800000000036, 0.0, -1.7240000000029454, -1.155999999991053, ... ], [ -1.2560000000050309, -0.8880000000033306, -1.4840000000049258, 1.3520000000077914, 1.324000000000325, -1.24800000000036, -1.6639999999989996, -1.7240000000029454, ... ], [ 0.0, -1.2559999999872673, -0.8880000000033306, 0.0, 1.3519999999989096, 1.324000000000325, 0.0, -1.6639999999945587, ... ], [ 0.0, 0.0, 0.0, -0.8880000000033306, -1.483999999996044, 0.0, 1.324000000000325, -1.24800000000036, ... ], [ 0.0, 0.0, 0.0, -1.2559999999872673, -0.8880000000033306, -1.483999999996044, 1.3519999999989096, 1.324000000000325, ... ], ... ]
Measured Statistics: {meanExponent=-0.10972995230511605, negative=362, min=-2.0000000000042206, max=1.9800000000103068, mean=0.01614814814847541, count=1215, sum=19.620000000397624, positive=373, stdDev=0.9236712816193215, zeros=480}
Feedback Error: [ [ 3.248512570053208E-13, -3.5993430458347575E-13, 0.0, -2.9454216843305403E-12, 6.505906924303417E-14, 0.0, 0.0, 0.0, ... ], [ -9.972245251788081E-12, 3.248512570053208E-13, -3.5993430458347575E-13, 2.3204771437690397E-11, 5.936362512670712E-12, 6.505906924303417E-14, 0.0, 0.0, ... ], [ 0.0, 7.791323142214424E-12, 3.248512570053208E-13, 0.0, -3.44058115331336E-12, -2.9454216843305403E-12, 0.0, 0.0, ... ], [ -3.330558051573007E-12, -4.9258375156568945E-12, 0.0, 1.8088419651007825E-11, -3.5993430458347575E-13, 0.0, -2.9454216843305403E-12, 8.946843266244286E-12, ... ], [ -5.030864613786434E-12, -3.330558051573007E-12, -4.9258375156568945E-12, 7.791323142214424E-12, 3.248512570053208E-13, -3.5993430458347575E-13, 1.000310945187266E-12, -2.9454216843305403E-12, ... ], [ 0.0, 1.273270378021607E-11, -3.330558051573007E-12, 0.0, -1.0904610547868288E-12, 3.248512570053208E-13, 0.0, 5.441203043687892E-12, ... ], [ 0.0, 0.0, 0.0, -3.330558051573007E-12, 3.955946681344358E-12, 0.0, 3.248512570053208E-13, -3.5993430458347575E-13, ... ], [ 0.0, 0.0, 0.0, 1.273270378021607E-11, -3.330558051573007E-12, 3.955946681344358E-12, -1.0904610547868288E-12, 3.248512570053208E-13, ... ], ... ]
Error Statistics: {meanExponent=-11.611749585229138, negative=359, min=-2.1469492850201277E-11, max=2.371458585059827E-11, mean=3.27290500942472E-13, count=1215, sum=3.9765795864510345E-10, positive=376, stdDev=4.53529262988044E-12, zeros=480}

Returns

    {
      "absoluteTol" : {
        "count" : 1215,
        "sum" : 3.1548541799819585E-9,
        "min" : 0.0,
        "max" : 2.371458585059827E-11,
        "sumOfSquare" : 2.5121337947445844E-20,
        "standardDeviation" : 3.732791193570923E-12,
        "average" : 2.5965878024542868E-12
      },
      "relativeTol" : {
        "count" : 735,
        "sum" : 4.47973124334159E-9,
        "min" : 2.1466134835732244E-14,
        "max" : 2.63177934375787E-10,
        "sumOfSquare" : 3.3139556976268334E-19,
        "standardDeviation" : 2.034037502478333E-11,
        "average" : 6.094872439920531E-12
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 1215,
        "sum" : 3.1548541799819585E-9,
        "min" : 0.0,
        "max" : 2.371458585059827E-11,
        "sumOfSquare" : 2.5121337947445844E-20,
        "standardDeviation" : 3.732791193570923E-12,
        "average" : 2.5965878024542868E-12
      },
      "relativeTol" : {
        "count" : 735,
        "sum" : 4.47973124334159E-9,
        "min" : 2.1466134835732244E-14,
        "max" : 2.63177934375787E-10,
        "sumOfSquare" : 3.3139556976268334E-19,
        "standardDeviation" : 2.034037502478333E-11,
        "average" : 6.094872439920531E-12
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 2.5966e-12 +- 3.7328e-12 [0.0000e+00 - 2.3715e-11] (1215#)
relativeTol: 6.0949e-12 +- 2.0340e-11 [2.1466e-14 - 2.6318e-10] (735#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.08 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=2.5966e-12 +- 3.7328e-12 [0.0000e+00 - 2.3715e-11] (1215#), relativeTol=6.0949e-12 +- 2.0340e-11 [2.1466e-14 - 2.6318e-10] (735#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "2.690",
      "gc_time": "0.554"
    },
    "created_on": 1586747231864,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "IrregularGrid",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.IrregularGrid",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/IrregularGrid/derivativeTest/202004130711",
    "id": "81ce3c24-2ff7-4ad3-a984-dc9fb660ff92",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ConvolutionLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
      "javaDoc": ""
    }
  }