1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 3231269432764633088

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [ 0.08, 0.7, -0.128 ]
Inputs Statistics: {meanExponent=-0.7148673344486438, negative=1, min=-0.128, max=0.7, mean=0.2173333333333333, count=3, sum=0.6519999999999999, positive=2, stdDev=0.3517018939701949, zeros=0}
Output: [ 0.5683200969810486, -0.4964189856500527, 0.578722321524795 ]
Outputs Statistics: {meanExponent=-0.2623627899064362, negative=1, min=-0.4964189856500527, max=0.578722321524795, mean=0.21687447761859693, count=3, sum=0.6506234328557908, positive=2, stdDev=0.5043925224962552, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.02 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [ 0.08, 0.7, -0.128 ]
Value Statistics: {meanExponent=-0.7148673344486438, negative=1, min=-0.128, max=0.7, mean=0.2173333333333333, count=3, sum=0.6519999999999999, positive=2, stdDev=0.3517018939701949, zeros=0}
Implemented Feedback: [ [ 0.890653497785285, -0.1830731332087863, -0.022963157155914196 ], [ 0.7717175206684737, -0.6815256847356844, 0.6897359687274536 ], [ 0.33698786960707616, 0.03675902873727107, -0.7636265311566455 ] ]
Implemented Statistics: {meanExponent=-0.5434613602755067, negative=4, min=-0.7636265311566455, max=0.890653497785285, mean=0.11940726436316991, count=9, sum=1.0746653792685292, positive=5, stdDev=0.5708456013788513, zeros=0}
Measured Feedback: [ [ 0.8906534977859426, -0.18307313320864438, -0.02296315715577535 ], [ 0.7717175206689575, -0.6815256847353401, 0.6897359687274562 ], [ 0.3369878696068973, 0.0367590287375652, -0.7636265311561363 ] ]
Measured Statistics: {meanExponent=-0.543461360275466, negative=4, min=-0.7636265311561363, max=0.8906534977859426, mean=0.11940726436343585, count=9, sum=1.0746653792709226, positive=5, stdDev=0.570845601378846, zeros=0}
Feedback Error: [ [ 6.575850974854802E-13, 1.4191425812271063E-13, 1.3884726701718364E-13 ], [ 4.838351941316432E-13, 3.4428015993626104E-13, 2.6645352591003757E-15 ], [ -1.7885692926711272E-13, 2.941327736927235E-13, 5.091482790930968E-13 ] ]
Error Statistics: {meanExponent=-12.756928919827352, negative=1, min=-1.7885692926711272E-13, max=6.575850974854802E-13, mean=2.6595007060789853E-13, count=9, sum=2.3935506354710867E-12, positive=8, stdDev=2.506967409506111E-13, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 9,
        "sum" : 2.751264494005312E-12,
        "min" : 2.6645352591003757E-15,
        "max" : 6.575850974854802E-13,
        "sumOfSquare" : 1.2022046638164362E-24,
        "standardDeviation" : 2.0032028854964387E-13,
        "average" : 3.0569605488947913E-13
      },
      "relativeTol" : {
        "count" : 9,
        "sum" : 8.947575578478867E-12,
        "min" : 1.931561771395201E-15,
        "max" : 4.000823522756965E-12,
        "sumOfSquare" : 2.5776836969531023E-23,
        "standardDeviation" : 1.3695652369228358E-12,
        "average" : 9.941750642754298E-13
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.02 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Learning Gradient for weight setByCoord 0
Weights: [ 0.890653497785285, 0.7717175206684737, 0.33698786960707616, -0.1830731332087863, -0.6815256847356844, 0.03675902873727107, -0.022963157155914196, 0.6897359687274536, -0.7636265311566455 ]
Implemented Gradient: [ [ 0.08, 0.0, 0.0 ], [ 0.7, 0.0, 0.0 ], [ -0.128, 0.0, 0.0 ], [ 0.0, 0.08, 0.0 ], [ 0.0, 0.7, 0.0 ], [ 0.0, -0.128, 0.0 ], [ 0.0, 0.0, 0.08 ], [ 0.0, 0.0, 0.7 ], [ 0.0, 0.0, -0.128 ] ]
Implemented Statistics: {meanExponent=-0.7148673344486437, negative=3, min=-0.128, max=0.7, mean=0.07244444444444442, count=27, sum=1.9559999999999995, positive=6, stdDev=0.22743746936056028, zeros=18}
Measured Gradient: [ [ 0.08000000000008001, 0.0, 0.0 ], [ 0.700000000000145, 0.0, 0.0 ], [ -0.12800000000035006, 0.0, 0.0 ], [ 0.0, 0.08000000000008001, 0.0 ], [ 0.0, 0.700000000000145, 0.0 ], [ 0.0, -0.12799999999979494, 0.0 ], [ 0.0, 0.0, 0.08000000000008001 ], [ 0.0, 0.0, 0.700000000000145 ], [ 0.0, 0.0, -0.12800000000035006 ] ]
Measured Statistics: {meanExponent=-0.7148673344482823, negative=3, min=-0.12800000000035006, max=0.700000000000145, mean=0.0724444444444511, count=27, sum=1.9560000000001798, positive=6, stdDev=0.22743746936062123, zeros=18}
Gradient Error: [ [ 8.000544671205034E-14, 0.0, 0.0 ], [ 1.4499512701604544E-13, 0.0, 0.0 ], [ -3.5005331966431186E-13, 0.0, 0.0 ], [ 0.0, 8.000544671205034E-14, 0.0 ], [ 0.0, 1.4499512701604544E-13, 0.0 ], [ 0.0, 2.0505819264826641E-13, 0.0 ], [ 0.0, 0.0, 8.000544671205034E-14 ], [ 0.0, 0.0, 1.4499512701604544E-13 ], [ 0.0, 0.0, -3.5005331966431186E-13 ] ]
Error Statistics: {meanExponent=-12.822937287783278, negative=2, min=-3.5005331966431186E-13, max=2.0505819264826641E-13, mean=6.66493609273815E-15, count=27, sum=1.7995327450393006E-13, positive=7, stdDev=1.1677733062604698E-13, zeros=18}

Returns

    {
      "absoluteTol" : {
        "count" : 36,
        "sum" : 4.33143104716649E-12,
        "min" : 0.0,
        "max" : 6.575850974854802E-13,
        "sumOfSquare" : 1.5716015544906011E-24,
        "standardDeviation" : 1.7081946882606328E-13,
        "average" : 1.2031752908795804E-13
      },
      "relativeTol" : {
        "count" : 18,
        "sum" : 1.4294181672841468E-11,
        "min" : 1.931561771395201E-15,
        "max" : 4.000823522756965E-12,
        "sumOfSquare" : 3.0940275217569236E-23,
        "standardDeviation" : 1.0432045302993989E-12,
        "average" : 7.941212040467482E-13
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 1.2032e-13 +- 1.7082e-13 [0.0000e+00 - 6.5759e-13] (36#)
relativeTol: 7.9412e-13 +- 1.0432e-12 [1.9316e-15 - 4.0008e-12] (18#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.2032e-13 +- 1.7082e-13 [0.0000e+00 - 6.5759e-13] (36#), relativeTol=7.9412e-13 +- 1.0432e-12 [1.9316e-15 - 4.0008e-12] (18#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.174",
      "gc_time": "0.108"
    },
    "created_on": 1586736965444,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/FullyConnectedLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/FullyConnectedLayer/Basic/derivativeTest/202004131605",
    "id": "87747dcc-242b-4af4-878f-4ba304973c4c",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "FullyConnectedLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.FullyConnectedLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/FullyConnectedLayer.java",
      "javaDoc": ""
    }
  }