1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 4647928499251334144

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [ -0.608, 1.764 ]
Inputs Statistics: {meanExponent=0.015201080034267928, negative=1, min=-0.608, max=1.764, mean=0.5780000000000001, count=2, sum=1.1560000000000001, positive=1, stdDev=1.1860000000000002, zeros=0}
Output: [ -0.651392, -0.160448 ]
Outputs Statistics: {meanExponent=-0.49041163576836666, negative=2, min=-0.651392, max=-0.160448, mean=-0.40592, count=2, sum=-0.81184, positive=0, stdDev=0.245472, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.01 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [ -0.608, 1.764 ]
Value Statistics: {meanExponent=0.015201080034267928, negative=1, min=-0.608, max=1.764, mean=0.5780000000000001, count=2, sum=1.1560000000000001, positive=1, stdDev=1.1860000000000002, zeros=0}
Implemented Feedback: [ [ 0.7, 0.496 ], [ -0.128, 0.08 ] ]
Implemented Statistics: {meanExponent=-0.6122800817139334, negative=1, min=-0.128, max=0.7, mean=0.28700000000000003, count=4, sum=1.1480000000000001, positive=3, stdDev=0.32761410226057114, zeros=0}
Measured Feedback: [ [ 0.700000000000145, 0.4959999999998299 ], [ -0.12800000000035006, 0.08000000000008001 ] ]
Measured Statistics: {meanExponent=-0.6122800817135428, negative=1, min=-0.12800000000035006, max=0.700000000000145, mean=0.2869999999999262, count=4, sum=1.1479999999997048, positive=3, stdDev=0.327614102260688, zeros=0}
Feedback Error: [ [ 1.4499512701604544E-13, -1.7008616737257398E-13 ], [ -3.5005331966431186E-13, 8.000544671205034E-14 ] ]
Error Statistics: {meanExponent=-12.790180960785424, negative=2, min=-3.5005331966431186E-13, max=1.4499512701604544E-13, mean=-7.378472832719751E-14, count=4, sum=-2.9513891330879005E-13, positive=2, stdDev=1.9818825946824943E-13, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 4,
        "sum" : 7.451400607649816E-13,
        "min" : 8.000544671205034E-14,
        "max" : 3.5005331966431186E-13,
        "sumOfSquare" : 1.7889108930149003E-25,
        "standardDeviation" : 1.0010327387538319E-13,
        "average" : 1.862850151912454E-13
      },
      "relativeTol" : {
        "count" : 4,
        "sum" : 2.142455599768211E-12,
        "min" : 1.0356794786859316E-13,
        "max" : 1.3673957799368484E-12,
        "sumOfSquare" : 2.1599293693963977E-24,
        "standardDeviation" : 5.030905410937123E-13,
        "average" : 5.356138999420527E-13
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.01 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Learning Gradient for weight setByCoord 0
Weights: [ 0.7, -0.128, 0.496, 0.08 ]
Implemented Gradient: [ [ -0.608, 0.0 ], [ 1.764, 0.0 ], [ 0.0, -0.608 ], [ 0.0, 1.764 ] ]
Implemented Statistics: {meanExponent=0.015201080034267928, negative=2, min=-0.608, max=1.764, mean=0.28900000000000003, count=8, sum=2.3120000000000003, positive=2, stdDev=0.8870281844451168, zeros=4}
Measured Gradient: [ [ -0.6079999999997199, 0.0 ], [ 1.7639999999996547, 0.0 ], [ 0.0, -0.6079999999997199 ], [ 0.0, 1.7640000000002098 ] ]
Measured Statistics: {meanExponent=0.015201080034159556, negative=2, min=-0.6079999999997199, max=1.7640000000002098, mean=0.2890000000000531, count=8, sum=2.312000000000425, positive=2, stdDev=0.8870281844450177, zeros=4}
Gradient Error: [ [ 2.80109269112927E-13, 0.0 ], [ -3.452793606584237E-13, 0.0 ], [ 0.0, 2.80109269112927E-13 ], [ 0.0, 2.0983215165415459E-13 ] ]
Error Statistics: {meanExponent=-12.561325596632196, negative=1, min=-3.452793606584237E-13, max=2.80109269112927E-13, mean=5.309641615269811E-14, count=8, sum=4.247713292215849E-13, positive=3, stdDev=1.9287818007241456E-13, zeros=4}

Returns

    {
      "absoluteTol" : {
        "count" : 12,
        "sum" : 1.860470111303414E-12,
        "min" : 0.0,
        "max" : 3.5005331966431186E-13,
        "sumOfSquare" : 4.990608633519483E-25,
        "standardDeviation" : 1.3248116546350934E-13,
        "average" : 1.5503917594195116E-13
      },
      "relativeTol" : {
        "count" : 8,
        "sum" : 2.7605061674619497E-12,
        "min" : 5.947623346205842E-14,
        "max" : 1.3673957799368484E-12,
        "sumOfSquare" : 2.27917002100766E-24,
        "standardDeviation" : 4.0721934099346684E-13,
        "average" : 3.450632709327437E-13
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 1.5504e-13 +- 1.3248e-13 [0.0000e+00 - 3.5005e-13] (12#)
relativeTol: 3.4506e-13 +- 4.0722e-13 [5.9476e-14 - 1.3674e-12] (8#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.5504e-13 +- 1.3248e-13 [0.0000e+00 - 3.5005e-13] (12#), relativeTol=3.4506e-13 +- 4.0722e-13 [5.9476e-14 - 1.3674e-12] (8#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.152",
      "gc_time": "0.092"
    },
    "created_on": 1586736118447,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedReferenceLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayer/Basic/derivativeTest/202004130158",
    "id": "e4a653bd-e757-4e7f-b2da-f74cd4834a85",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "FullyConnectedReferenceLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedReferenceLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/FullyConnectedReferenceLayer.java",
      "javaDoc": ""
    }
  }