1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 5831804167832276992

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [
[ [ 0.08 ], [ -0.608 ], [ 1.208 ], [ -0.852 ] ],
[ [ 0.7 ], [ 1.764 ], [ -1.72 ], [ 1.912 ] ],
[ [ -0.128 ], [ 0.048 ], [ -1.028 ], [ -1.688 ] ],
[ [ 0.496 ], [ 1.524 ], [ -0.384 ], [ -0.804 ] ]
]
Inputs Statistics: {meanExponent=-0.20600101677773192, negative=8, min=-1.72, max=1.912, mean=0.0325, count=16, sum=0.52, positive=8, stdDev=1.1193595266937248, zeros=0}
Output: [
[ [ 0.0325000137090683 ] ]
]
Outputs Statistics: {meanExponent=-1.4881164558281577, negative=0, min=0.0325000137090683, max=0.0325000137090683, mean=0.0325000137090683, count=1, sum=0.0325000137090683, positive=1, stdDev=0.0, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.07 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [
[ [ 0.08 ], [ -0.608 ], [ 1.208 ], [ -0.852 ] ],
[ [ 0.7 ], [ 1.764 ], [ -1.72 ], [ 1.912 ] ],
[ [ -0.128 ], [ 0.048 ], [ -1.028 ], [ -1.688 ] ],
[ [ 0.496 ], [ 1.524 ], [ -0.384 ], [ -0.804 ] ]
]
Value Statistics: {meanExponent=-0.20600101677773192, negative=8, min=-1.72, max=1.912, mean=0.0325, count=16, sum=0.52, positive=8, stdDev=1.1193595266937248, zeros=0}
Implemented Feedback: [ [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], [ 0.0625 ], ... ]
Implemented Statistics: {meanExponent=-1.2041199826559248, negative=0, min=0.0625, max=0.0625, mean=0.0625, count=16, sum=1.0, positive=16, stdDev=0.0, zeros=0}
Measured Feedback: [ [ 0.06249547004699707 ], [ 0.06249547004699707 ], [ 0.06249547004699707 ], [ 0.06251037120819092 ], [ 0.06249547004699707 ], [ 0.06249547004699707 ], [ 0.06249547004699707 ], [ 0.06249547004699707 ], ... ]
Measured Statistics: {meanExponent=-1.2041320475650497, negative=0, min=0.06249547004699707, max=0.06251037120819092, mean=0.06249826401472092, count=16, sum=0.9999722242355347, positive=16, stdDev=5.8161076143369065E-6, zeros=0}
Feedback Error: [ [ -4.5299530029296875E-6 ], [ -4.5299530029296875E-6 ], [ -4.5299530029296875E-6 ], [ 1.0371208190917969E-5 ], [ -4.5299530029296875E-6 ], [ -4.5299530029296875E-6 ], [ -4.5299530029296875E-6 ], [ -4.5299530029296875E-6 ], ... ]
Error Statistics: {meanExponent=-5.276455868154418, negative=13, min=-4.5299530029296875E-6, max=1.0371208190917969E-5, mean=-1.735985279083252E-6, count=16, sum=-2.777576446533203E-5, positive=3, stdDev=5.8161076143369065E-6, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 16,
        "sum" : 9.000301361083984E-5,
        "min" : 4.5299530029296875E-6,
        "max" : 1.0371208190917969E-5,
        "sumOfSquare" : 5.894520427318639E-10,
        "standardDeviation" : 2.2799141848200674E-6,
        "average" : 5.62518835067749E-6
      },
      "relativeTol" : {
        "count" : 16,
        "sum" : 7.200205323571488E-4,
        "min" : 3.624093738138246E-5,
        "max" : 8.296278213305896E-5,
        "sumOfSquare" : 3.772274170742892E-8,
        "standardDeviation" : 1.8236114184731284E-5,
        "average" : 4.50012832723218E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 16,
        "sum" : 9.000301361083984E-5,
        "min" : 4.5299530029296875E-6,
        "max" : 1.0371208190917969E-5,
        "sumOfSquare" : 5.894520427318639E-10,
        "standardDeviation" : 2.2799141848200674E-6,
        "average" : 5.62518835067749E-6
      },
      "relativeTol" : {
        "count" : 16,
        "sum" : 7.200205323571488E-4,
        "min" : 3.624093738138246E-5,
        "max" : 8.296278213305896E-5,
        "sumOfSquare" : 3.772274170742892E-8,
        "standardDeviation" : 1.8236114184731284E-5,
        "average" : 4.50012832723218E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 5.6252e-06 +- 2.2799e-06 [4.5300e-06 - 1.0371e-05] (16#)
relativeTol: 4.5001e-05 +- 1.8236e-05 [3.6241e-05 - 8.2963e-05] (16#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.02 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=5.6252e-06 +- 2.2799e-06 [4.5300e-06 - 1.0371e-05] (16#), relativeTol=4.5001e-05 +- 1.8236e-05 [3.6241e-05 - 8.2963e-05] (16#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.256",
      "gc_time": "0.126"
    },
    "created_on": 1586744465722,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Float",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.BandAvgReducerLayerTest.Float",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/BandAvgReducerLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/BandAvgReducerLayer/Float/derivativeTest/202004132105",
    "id": "18100626-9625-4c5d-9b9f-06ee10370bc5",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "BandAvgReducerLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.BandAvgReducerLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/BandAvgReducerLayer.java",
      "javaDoc": ""
    }
  }