1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 133305240581897216

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Inputs Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Output: [
[ [ 0.21776758783465988, 0.10944568329718811, 0.672786728868152 ], [ 0.38476222787786835, 0.4588051240527918, 0.15643264806933976 ] ],
[ [ 0.2508238804179965, 0.7268724544377955, 0.02230366514420795 ], [ 0.23751546211083552, 0.6639671829454358, 0.09851735494372868 ] ]
]
Outputs Statistics: {meanExponent=-0.6294356703587121, negative=0, min=0.02230366514420795, max=0.7268724544377955, mean=0.3333333333333333, count=12, sum=4.0, positive=12, stdDev=0.23460078463826506, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.18 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Value Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Implemented Feedback: [ [ 0.17034486552333358, 0.0, 0.0, 0.0, -0.02383372245054478, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.1879112614300551, 0.0, 0.0, 0.0, -0.18231696959104124, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.23672025587632764, 0.0, 0.0, 0.0, -0.1765308816923339, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.18110186736911177, 0.0, 0.0, 0.0, -0.15770247228371484, ... ], [ -0.02383372245054478, 0.0, 0.0, 0.0, 0.09746732570479971, 0.0, 0.0, 0.0, ... ], [ 0.0, -0.18231696959104124, 0.0, 0.0, 0.0, 0.19852888941737043, 0.0, 0.0, ... ], [ 0.0, 0.0, -0.1765308816923339, 0.0, 0.0, 0.0, 0.24830298219569416, 0.0, ... ], [ 0.0, 0.0, 0.0, -0.15770247228371484, 0.0, 0.0, 0.0, 0.223114762916938, ... ], ... ]
Implemented Statistics: {meanExponent=-1.1193075872992841, negative=24, min=-0.18231696959104124, max=0.24830298219569416, mean=1.5419764230904951E-18, count=144, sum=2.220446049250313E-16, positive=12, stdDev=0.06713358813205941, zeros=108}
Measured Feedback: [ [ 0.17034967320112004, 0.0, 0.0, 0.0, -0.02383439511463581, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.1879159436901645, 0.0, 0.0, 0.0, -0.1823215124552302, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.23672298362198418, 0.0, 0.0, 0.0, -0.17653291587038655, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.18110662098669295, 0.0, 0.0, 0.0, -0.15770661170644296, ... ], [ -0.023834653303522746, 0.0, 0.0, 0.0, 0.09747113240018979, 0.0, 0.0, 0.0, ... ], [ 0.0, -0.1823128332628654, 0.0, 0.0, 0.0, 0.198524385280896, 0.0, 0.0, ... ], [ 0.0, 0.0, -0.17653160876540674, 0.0, 0.0, 0.0, 0.2483040048745222, 0.0, ... ], [ 0.0, 0.0, 0.0, -0.1576998863916934, 0.0, 0.0, 0.0, 0.22311110444150906, ... ], ... ]
Measured Statistics: {meanExponent=-1.1193003492407743, negative=24, min=-0.1823215124552302, max=0.2483040048745222, mean=2.9875793197378345E-14, count=144, sum=4.3021142204224816E-12, positive=12, stdDev=0.06713399459226534, zeros=108}
Feedback Error: [ [ 4.807677786461406E-6, 0.0, 0.0, 0.0, -6.726640910288684E-7, 0.0, 0.0, 0.0, ... ], [ 0.0, 4.682260109412084E-6, 0.0, 0.0, 0.0, -4.5428641889666554E-6, 0.0, 0.0, ... ], [ 0.0, 0.0, 2.7277456565444336E-6, 0.0, 0.0, 0.0, -2.0341780526400566E-6, 0.0, ... ], [ 0.0, 0.0, 0.0, 4.753617581176872E-6, 0.0, 0.0, 0.0, -4.139422728116626E-6, ... ], [ -9.308529779657182E-7, 0.0, 0.0, 0.0, 3.806695390073944E-6, 0.0, 0.0, 0.0, ... ], [ 0.0, 4.136328175835535E-6, 0.0, 0.0, 0.0, -4.504136474431375E-6, 0.0, 0.0, ... ], [ 0.0, 0.0, -7.270730728314856E-7, 0.0, 0.0, 0.0, 1.0226788280387478E-6, 0.0, ... ], [ 0.0, 0.0, 0.0, 2.5858920214283376E-6, 0.0, 0.0, 0.0, -3.6584754289359367E-6, ... ], ... ]
Error Statistics: {meanExponent=-5.772543849566124, negative=21, min=-4.5428641889666554E-6, max=4.807677786461406E-6, mean=2.987449215477136E-14, count=144, sum=4.301926870287076E-12, positive=15, stdDev=1.4315813238043176E-6, zeros=108}

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 8.581686855266973E-5,
        "min" : 0.0,
        "max" : 4.807677786461406E-6,
        "sumOfSquare" : 2.9511721247980656E-10,
        "standardDeviation" : 1.3016405482117312E-6,
        "average" : 5.959504760602065E-7
      },
      "relativeTol" : {
        "count" : 36,
        "sum" : 4.6908620136906526E-4,
        "min" : 2.0593299271323702E-6,
        "max" : 2.3884969562578075E-5,
        "sumOfSquare" : 7.453358845075728E-9,
        "standardDeviation" : 6.103470862117076E-6,
        "average" : 1.3030172260251813E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 8.581686855266973E-5,
        "min" : 0.0,
        "max" : 4.807677786461406E-6,
        "sumOfSquare" : 2.9511721247980656E-10,
        "standardDeviation" : 1.3016405482117312E-6,
        "average" : 5.959504760602065E-7
      },
      "relativeTol" : {
        "count" : 36,
        "sum" : 4.6908620136906526E-4,
        "min" : 2.0593299271323702E-6,
        "max" : 2.3884969562578075E-5,
        "sumOfSquare" : 7.453358845075728E-9,
        "standardDeviation" : 6.103470862117076E-6,
        "average" : 1.3030172260251813E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 5.9595e-07 +- 1.3016e-06 [0.0000e+00 - 4.8077e-06] (144#)
relativeTol: 1.3030e-05 +- 6.1035e-06 [2.0593e-06 - 2.3885e-05] (36#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=5.9595e-07 +- 1.3016e-06 [0.0000e+00 - 4.8077e-06] (144#), relativeTol=1.3030e-05 +- 6.1035e-06 [2.0593e-06 - 2.3885e-05] (36#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.365",
      "gc_time": "0.130"
    },
    "created_on": 1586742606030,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Pixel",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.SoftmaxLayerTest.Pixel",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/SoftmaxLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/SoftmaxActivationLayer/Pixel/derivativeTest/202004135006",
    "id": "2b18c667-a759-4dc6-87c1-8fa7f3889f5b",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "SoftmaxActivationLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.SoftmaxActivationLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/SoftmaxActivationLayer.java",
      "javaDoc": ""
    }
  }