1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging 
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine8005764550548145692/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 8919857194981494784

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -1.72, -0.712 ], [ 0.496, -0.852, 1.356 ], [ 0.048, -0.804, 1.032 ] ],
[ [ 0.7, -1.028, 1.048 ], [ -0.608, 1.912, 1.512 ], [ 1.524, 1.108, 0.3 ] ],
[ [ -0.128, -0.384, -1.616 ], [ 1.764, -1.688, 1.556 ], [ 1.208, 0.028, 0.636 ] ]
]
Inputs Statistics: {meanExponent=-0.18240580381395474, negative=10, min=-1.72, max=1.912, mean=0.2506666666666667, count=27, sum=6.768000000000001, positive=17, stdDev=1.0872753578113061, zeros=0}
Output: [
[ [ -6.571312000000002, 5.496432, 5.235648 ], [ 0.6125759999999997, 0.6574080000000001, 7.769311999999999 ], [ 5.255040000000001, -1.7711679999999994, 5.89656 ] ],
[ [ 5.552447999999999, -3.9092000000000002, -1.652784 ], [ -9.740128000000002, 11.555087999999998, 0.2526880000000016 ], [ -7.360544000000001, 15.581392000000001, 7.340688000000001 ] ],
[ [ 1.42288, 7.388927999999999, 2.842463999999999 ], [ -7.102847999999999, -0.055887999999999896, -2.7394559999999997 ], [ 0.9809600000000002, 3.458848, 9.224688 ] ]
]
Outputs Statistics: {meanExponent=0.4982545904891352, negative=9, min=-9.740128000000002, max=15.581392000000001, mean=2.0600266666666665, count=27, sum=55.62072, positive=18, stdDev=5.968236976590286, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.91 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -1.72, -0.712 ], [ 0.496, -0.852, 1.356 ], [ 0.048, -0.804, 1.032 ] ],
[ [ 0.7, -1.028, 1.048 ], [ -0.608, 1.912, 1.512 ], [ 1.524, 1.108, 0.3 ] ],
[ [ -0.128, -0.384, -1.616 ], [ 1.764, -1.688, 1.556 ], [ 1.208, 0.028, 0.636 ] ]
]
Value Statistics: {meanExponent=-0.18240580381395474, negative=10, min=-1.72, max=1.912, mean=0.2506666666666667, count=27, sum=6.768000000000001, positive=17, stdDev=1.0872753578113061, zeros=0}
Implemented Feedback: [ [ 0.148, 1.64, 0.0, -0.408, -1.476, 0.0, 0.0, 0.0, ... ], [ -0.892, 0.148, 1.64, -0.068, -0.408, -1.476, 0.0, 0.0, ... ], [ 0.0, -0.892, 0.148, 0.0, -0.068, -0.408, 0.0, 0.0, ... ], [ -1.832, -0.636, 0.0, 0.148, 1.64, 0.0, -0.408, -1.476, ... ], [ -1.54, -1.832, -0.636, -0.892, 0.148, 1.64, -0.068, -0.408, ... ], [ 0.0, -1.54, -1.832, 0.0, -0.892, 0.148, 0.0, -0.068, ... ], [ 0.0, 0.0, 0.0, -1.832, -0.636, 0.0, 0.148, 1.64, ... ], [ 0.0, 0.0, 0.0, -1.54, -1.832, -0.636, -0.892, 0.148, ... ], ... ]
Implemented Statistics: {meanExponent=-0.16443218578341598, negative=219, min=-2.0, max=1.98, mean=0.04494924554183821, count=729, sum=32.76800000000006, positive=222, stdDev=0.9020990482742104, zeros=288}
Measured Feedback: [ [ 0.14800000002423985, 1.6400000000071913, 0.0, -0.40799999999840963, -1.4760000000002549, 0.0, 0.0, 0.0, ... ], [ -0.8919999999790207, 0.14800000000647628, 1.6399999999983095, -0.06800000000417583, -0.4080000000072914, -1.475999999991373, 0.0, 0.0, ... ], [ 0.0, -0.8919999999967843, 0.1479999999975945, 0.0, -0.06799999999529405, -0.40799999999840963, 0.0, 0.0, ... ], [ -1.831999999986067, -0.6359999999983046, 0.0, 0.1479999999975945, 1.6399999999983095, 0.0, -0.4080000000072914, -1.4760000000002549, ... ], [ -1.5399999999843317, -1.8319999999949488, -0.6359999999983046, -0.8919999999967843, 0.14800000000647628, 1.6399999999983095, -0.06800000000417583, -0.40799999999840963, ... ], [ 0.0, -1.5400000000020952, -1.8320000000038306, 0.0, -0.8919999999967843, 0.14800000000647628, 0.0, -0.06799999999529405, ... ], [ 0.0, 0.0, 0.0, -1.8320000000038306, -0.6359999999894228, 0.0, 0.1479999999975945, 1.6399999999983095, ... ], [ 0.0, 0.0, 0.0, -1.5400000000020952, -1.831999999986067, -0.6359999999983046, -0.892000000005666, 0.1479999999975945, ... ], ... ]
Measured Statistics: {meanExponent=-0.16443218578556595, negative=219, min=-1.9999999999997797, max=1.980000000001425, mean=0.0449492455416283, count=729, sum=32.76799999984703, positive=222, stdDev=0.902099048273771, zeros=288}
Feedback Error: [ [ 2.4239860119124046E-11, 7.191358619706989E-12, 0.0, 1.5903389716243055E-12, -2.5490720645393594E-13, 0.0, 0.0, 0.0, ... ], [ 2.097932938482927E-11, 6.476291725121541E-12, -1.6904255772942633E-12, -4.17582635137137E-12, -7.291445225376947E-12, 8.626876990547316E-12, 0.0, 0.0, ... ], [ 0.0, 3.215760990826766E-12, -2.405492471879711E-12, 0.0, 4.705957845629882E-12, 1.5903389716243055E-12, 0.0, 0.0, ... ], [ 1.393307691444079E-11, 1.6954215809050766E-12, 0.0, -2.405492471879711E-12, -1.6904255772942633E-12, 0.0, -7.291445225376947E-12, -2.5490720645393594E-13, ... ], [ 1.566835550192991E-11, 5.051292717439537E-12, 1.6954215809050766E-12, 3.215760990826766E-12, 6.476291725121541E-12, -1.6904255772942633E-12, -4.17582635137137E-12, 1.5903389716243055E-12, ... ], [ 0.0, -2.0952128920725954E-12, -3.830491479561715E-12, 0.0, 3.215760990826766E-12, 6.476291725121541E-12, 0.0, 4.705957845629882E-12, ... ], [ 0.0, 0.0, 0.0, -3.830491479561715E-12, 1.0577205777906329E-11, 0.0, -2.405492471879711E-12, -1.6904255772942633E-12, ... ], [ 0.0, 0.0, 0.0, -2.0952128920725954E-12, 1.393307691444079E-11, 1.6954215809050766E-12, -5.6660232061744864E-12, -2.405492471879711E-12, ... ], ... ]
Error Statistics: {meanExponent=-11.706892712029358, negative=231, min=-3.29865024184528E-11, max=2.4239860119124046E-11, mean=-2.0983379833268048E-13, count=729, sum=-1.5296883898452407E-10, positive=210, stdDev=5.058723712869992E-12, zeros=288}

Returns

    {
      "absoluteTol" : {
        "count" : 729,
        "sum" : 1.804928852211507E-9,
        "min" : 0.0,
        "max" : 3.29865024184528E-11,
        "sumOfSquare" : 1.868770783720932E-20,
        "standardDeviation" : 4.416406942796128E-12,
        "average" : 2.4758969166138642E-12
      },
      "relativeTol" : {
        "count" : 441,
        "sum" : 3.4904053551003534E-9,
        "min" : 2.1474680271574497E-14,
        "max" : 2.9193357792082866E-10,
        "sumOfSquare" : 3.1783180579532165E-19,
        "standardDeviation" : 2.5652753328589614E-11,
        "average" : 7.914751372109646E-12
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.58 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Learning Gradient for weight setByCoord 0
Weights: [ -1.54, -1.832, -0.636, -0.892, 0.148, 1.64, -0.068, -0.408, ... ]
Implemented Gradient: [ [ -0.608, 1.764, 0.0, 1.524, 1.208, 0.0, 0.0, 0.0, ... ], [ 0.496, -0.608, 1.764, 0.048, 1.524, 1.208, 0.0, 0.0, ... ], [ 0.0, 0.496, -0.608, 0.0, 0.048, 1.524, 0.0, 0.0, ... ], [ 0.7, -0.128, 0.0, -0.608, 1.764, 0.0, 1.524, 1.208, ... ], [ 0.08, 0.7, -0.128, 0.496, -0.608, 1.764, 0.048, 1.524, ... ], [ 0.0, 0.08, 0.7, 0.0, 0.496, -0.608, 0.0, 0.048, ... ], [ 0.0, 0.0, 0.0, 0.7, -0.128, 0.0, -0.608, 1.764, ... ], [ 0.0, 0.0, 0.0, 0.08, 0.7, -0.128, 0.496, -0.608, ... ], ... ]
Implemented Statistics: {meanExponent=-0.12559721447679753, negative=153, min=-1.72, max=1.912, mean=0.07368998628257888, count=2187, sum=161.16, positive=288, stdDev=0.5198601874099508, zeros=1746}
Measured Gradient: [ [ -0.6079999999997199, 1.7639999999996547, 0.0, 1.5240000000016352, 1.2080000000125324, 0.0, 0.0, 0.0, ... ], [ 0.496000000005381, -0.6079999999997199, 1.7639999999996547, 0.04800000000138027, 1.524000000010517, 1.2080000000036506, 0.0, 0.0, ... ], [ 0.0, 0.496000000005381, -0.6079999999997199, 0.0, 0.048000000010262056, 1.5240000000016352, 0.0, 0.0, ... ], [ 0.7000000000090267, -0.12799999999479894, 0.0, -0.6079999999997199, 1.7640000000085365, 0.0, 1.5239999999927534, 1.2080000000036506, ... ], [ 0.08000000001118224, 0.700000000000145, -0.12800000000368073, 0.49599999999649924, -0.6079999999997199, 1.7639999999996547, 0.04799999999249849, 1.5240000000016352, ... ], [ 0.0, 0.08000000000230045, 0.700000000000145, 0.0, 0.496000000005381, -0.6079999999997199, 0.0, 0.04800000000138027, ... ], [ 0.0, 0.0, 0.0, 0.700000000000145, -0.12799999998591716, 0.0, -0.6079999999997199, 1.7639999999996547, ... ], [ 0.0, 0.0, 0.0, 0.08000000000230045, 0.7000000000090267, -0.12799999999479894, 0.49599999999649924, -0.6079999999997199, ... ], ... ]
Measured Statistics: {meanExponent=-0.12559721447690333, negative=153, min=-1.7200000000272553, max=1.9120000000150128, mean=0.0736899862825276, count=2187, sum=161.15999999988784, positive=288, stdDev=0.5198601874097911, zeros=1746}
Gradient Error: [ [ 2.80109269112927E-13, -3.452793606584237E-13, 0.0, 1.6351364706679306E-12, 1.2532419546573692E-11, 0.0, 0.0, 0.0, ... ], [ 5.381028955753209E-12, 2.80109269112927E-13, -3.452793606584237E-13, 1.380270897577418E-12, 1.0516920667669183E-11, 3.65063534957244E-12, 0.0, 0.0, ... ], [ 0.0, 5.381028955753209E-12, 2.80109269112927E-13, 0.0, 1.026205509457867E-11, 1.6351364706679306E-12, 0.0, 0.0, ... ], [ 9.026779324017298E-12, 5.201061803461471E-12, 0.0, 2.80109269112927E-13, 8.536504836342829E-12, 0.0, -7.246647726333322E-12, 3.65063534957244E-12, ... ], [ 1.1182235692963616E-11, 1.4499512701604544E-13, -3.6807223935397815E-12, -3.5007552412480436E-12, 2.80109269112927E-13, -3.452793606584237E-13, -7.501513299423834E-12, 1.6351364706679306E-12, ... ], [ 0.0, 2.3004514959623634E-12, 1.4499512701604544E-13, 0.0, 5.381028955753209E-12, 2.80109269112927E-13, 0.0, 1.380270897577418E-12, ... ], [ 0.0, 0.0, 0.0, 1.4499512701604544E-13, 1.4082846000462723E-11, 0.0, 2.80109269112927E-13, -3.452793606584237E-13, ... ], [ 0.0, 0.0, 0.0, 2.3004514959623634E-12, 9.026779324017298E-12, 5.201061803461471E-12, -3.5007552412480436E-12, 2.80109269112927E-13, ... ], ... ]
Error Statistics: {meanExponent=-11.73697747259509, negative=217, min=-3.01461078322518E-11, max=1.631805801594055E-11, mean=-5.13127809307576E-14, count=2187, sum=-1.1222105189556686E-10, positive=224, stdDev=2.783474768728835E-12, zeros=1746}

Returns

    {
      "absoluteTol" : {
        "count" : 2916,
        "sum" : 3.5416438601276745E-9,
        "min" : 0.0,
        "max" : 3.29865024184528E-11,
        "sumOfSquare" : 3.56377556321452E-20,
        "standardDeviation" : 3.278156107895627E-12,
        "average" : 1.2145555075883658E-12
      },
      "relativeTol" : {
        "count" : 882,
        "sum" : 6.75336463044522E-9,
        "min" : 2.1474680271574497E-14,
        "max" : 2.9193357792082866E-10,
        "sumOfSquare" : 5.661173109320336E-19,
        "standardDeviation" : 2.4150126532765084E-11,
        "average" : 7.65687599823721E-12
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 1.2146e-12 +- 3.2782e-12 [0.0000e+00 - 3.2987e-11] (2916#)
relativeTol: 7.6569e-12 +- 2.4150e-11 [2.1475e-14 - 2.9193e-10] (882#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.02 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.2146e-12 +- 3.2782e-12 [0.0000e+00 - 3.2987e-11] (2916#), relativeTol=7.6569e-12 +- 2.4150e-11 [2.1475e-14 - 2.9193e-10] (882#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "1.984",
      "gc_time": "0.289"
    },
    "created_on": 1586748096167,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "NoPadding",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.NoPadding",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/NoPadding/derivativeTest/202004132136",
    "id": "602e6c29-af29-461b-aa42-73eb53ce5e8e",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ConvolutionLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
      "javaDoc": ""
    }
  }