1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 3044183577899240448

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [ 0.987408288993747, 0.20472560516257754, 0.44818654473814834, 0.7614510484920741 ],
[ 0.5206620191450972, 0.7992082822891965, 0.18012167911239696, 0.40188000183298456 ]
Inputs Statistics: {meanExponent=-0.29030756614172587, negative=0, min=0.20472560516257754, max=0.987408288993747, mean=0.6004428718466368, count=4, sum=2.401771487386547, positive=4, stdDev=0.2980939496664579, zeros=0},
{meanExponent=-0.3802804358624683, negative=0, min=0.18012167911239696, max=0.7992082822891965, mean=0.4754679955949188, count=4, sum=1.9018719823796753, positive=4, stdDev=0.22332426675086586, zeros=0}
Output: [ 1.528289684656872 ]
Outputs Statistics: {meanExponent=0.1842056818076238, negative=0, min=1.528289684656872, max=1.528289684656872, mean=1.528289684656872, count=1, sum=1.528289684656872, positive=1, stdDev=0.0, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.02 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [ 0.987408288993747, 0.20472560516257754, 0.44818654473814834, 0.7614510484920741 ]
Value Statistics: {meanExponent=-0.29030756614172587, negative=0, min=0.20472560516257754, max=0.987408288993747, mean=0.6004428718466368, count=4, sum=2.401771487386547, positive=4, stdDev=0.2980939496664579, zeros=0}
Implemented Feedback: [ [ -0.5273016491239871 ], [ -3.9038022706271933 ], [ -0.40188997467032955 ], [ -0.5277817958604698 ] ]
Implemented Statistics: {meanExponent=-0.08997286972074243, negative=4, min=-3.9038022706271933, max=-0.40188997467032955, mean=-1.340193922570495, count=4, sum=-5.36077569028198, positive=0, stdDev=1.4809886370871963, zeros=0}
Measured Feedback: [ [ -0.5273016467910452 ], [ -3.9038021704129733 ], [ -0.4018899657509678 ], [ -0.5277817960447351 ] ]
Measured Statistics: {meanExponent=-0.08997287536001707, negative=4, min=-3.9038021704129733, max=-0.4018899657509678, mean=-1.3401938947499303, count=4, sum=-5.360775578999721, positive=0, stdDev=1.4809885954268147, zeros=0}
Feedback Error: [ [ 2.3329419507334137E-9 ], [ 1.002142200334788E-7 ], [ 8.919361749182286E-9 ], [ -1.842652697092717E-10 ] ]
Error Statistics: {meanExponent=-8.35384736280098, negative=1, min=-1.842652697092717E-10, max=1.002142200334788E-7, mean=2.7820564615921306E-8, count=4, sum=1.1128225846368522E-7, positive=3, stdDev=4.192846950632643E-8, zeros=0}
Feedback for input 1
Inputs Values: [ 0.5206620191450972, 0.7992082822891965, 0.18012167911239696, 0.40188000183298456 ]
Value Statistics: {meanExponent=-0.3802804358624683, negative=0, min=0.18012167911239696, max=0.7992082822891965, mean=0.4754679955949188, count=4, sum=1.9018719823796753, positive=4, stdDev=0.22332426675086586, zeros=0}
Implemented Feedback: [ [ 0.012671658424788857 ], [ 1.5860847078523443 ], [ 0.8025457387304904 ], [ 0.2725293917176603 ] ]
Implemented Statistics: {meanExponent=-0.5892392561138665, negative=0, min=0.012671658424788857, max=1.5860847078523443, mean=0.6684578741813209, count=4, sum=2.6738314967252834, positive=4, stdDev=0.6014219577997455, zeros=0}
Measured Feedback: [ [ 0.01267166371832218 ], [ 1.5860847080162443 ], [ 0.8025457409743808 ], [ 0.2725293990479827 ] ]
Measured Statistics: {meanExponent=-0.5892392075225594, negative=0, min=0.01267166371832218, max=1.5860847080162443, mean=0.6684578779392325, count=4, sum=2.67383151175693, positive=4, stdDev=0.6014219553378995, zeros=0}
Feedback Error: [ [ 5.293533321751309E-9 ], [ 1.639000046793626E-10 ], [ 2.2438904068167176E-9 ], [ 7.330322404630607E-9 ] ]
Error Statistics: {meanExponent=-8.711387666152596, negative=0, min=1.639000046793626E-10, max=7.330322404630607E-9, mean=3.757911534469499E-9, count=4, sum=1.5031646137877996E-8, positive=4, stdDev=2.7536082033426276E-9, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 8,
        "sum" : 1.2668243514098176E-7,
        "min" : 1.639000046793626E-10,
        "max" : 1.002142200334788E-7,
        "sumOfSquare" : 1.021473851172005E-14,
        "standardDeviation" : 3.203256856322936E-8,
        "average" : 1.583530439262272E-8
      },
      "relativeTol" : {
        "count" : 8,
        "sum" : 2.5009018924062113E-7,
        "min" : 5.1668112005450464E-11,
        "max" : 2.0887290513292972E-7,
        "sumOfSquare" : 4.410352609977696E-14,
        "standardDeviation" : 6.734740924441317E-8,
        "average" : 3.126127365507764E-8
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 8,
        "sum" : 1.2668243514098176E-7,
        "min" : 1.639000046793626E-10,
        "max" : 1.002142200334788E-7,
        "sumOfSquare" : 1.021473851172005E-14,
        "standardDeviation" : 3.203256856322936E-8,
        "average" : 1.583530439262272E-8
      },
      "relativeTol" : {
        "count" : 8,
        "sum" : 2.5009018924062113E-7,
        "min" : 5.1668112005450464E-11,
        "max" : 2.0887290513292972E-7,
        "sumOfSquare" : 4.410352609977696E-14,
        "standardDeviation" : 6.734740924441317E-8,
        "average" : 3.126127365507764E-8
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 1.5835e-08 +- 3.2033e-08 [1.6390e-10 - 1.0021e-07] (8#)
relativeTol: 3.1261e-08 +- 6.7347e-08 [5.1668e-11 - 2.0887e-07] (8#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.00 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.5835e-08 +- 3.2033e-08 [1.6390e-10 - 1.0021e-07] (8#), relativeTol=3.1261e-08 +- 6.7347e-08 [5.1668e-11 - 2.0887e-07] (8#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.168",
      "gc_time": "0.112"
    },
    "created_on": 1586735767346,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.EntropyLossLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/EntropyLossLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/EntropyLossLayer/Basic/derivativeTest/202004125607",
    "id": "df14f8c6-5ebe-4af9-98c7-80b88fdb54b9",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "EntropyLossLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.EntropyLossLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/EntropyLossLayer.java",
      "javaDoc": ""
    }
  }