1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 9203828650370176000

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [ 8.0, 70.0, -12.8, 49.6 ]
Inputs Statistics: {meanExponent=1.3877199182860667, negative=1, min=-12.8, max=70.0, mean=28.700000000000003, count=4, sum=114.80000000000001, positive=3, stdDev=32.761410226057116, zeros=0}
Output: [ 0.06968641114982578, 0.6097560975609756, -0.11149825783972125, 0.43205574912891986 ]
Outputs Statistics: {meanExponent=-0.6722219697758881, negative=1, min=-0.11149825783972125, max=0.6097560975609756, mean=0.25, count=4, sum=1.0, positive=3, stdDev=0.28537813785764043, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.02 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [ 8.0, 70.0, -12.8, 49.6 ]
Value Statistics: {meanExponent=1.3877199182860667, negative=1, min=-12.8, max=70.0, mean=28.700000000000003, count=4, sum=114.80000000000001, positive=3, stdDev=32.761410226057116, zeros=0}
Implemented Feedback: [ [ 0.008103776906360403, -0.005311464264468429, 9.712391797885128E-4, -0.0037635518216804866 ], [ -6.070244873678204E-4, 0.0033993371292597944, 9.712391797885128E-4, -0.0037635518216804866 ], [ -6.070244873678204E-4, -0.005311464264468429, 0.009682040573516735, -0.0037635518216804866 ], [ -6.070244873678204E-4, -0.005311464264468429, 9.712391797885128E-4, 0.004947249572047736 ] ]
Implemented Statistics: {meanExponent=-2.604097116358875, negative=9, min=-0.005311464264468429, max=0.009682040573516735, mean=-1.0842021724855044E-19, count=16, sum=-1.734723475976807E-18, positive=7, stdDev=0.004517377272276163, zeros=0}
Measured Feedback: [ [ 0.008103769847250808, -0.0053114596376691026, 9.712383337856512E-4, -0.0037635485433673566 ], [ -6.070239585986847E-4, 0.0033993341685967238, 9.712383337856512E-4, -0.0037635485433673566 ], [ -6.070239584599069E-4, -0.0053114596365588795, 0.009682032139496366, -0.003763548542812245 ], [ -6.070239585986847E-4, -0.0053114596376691026, 9.712383337856512E-4, 0.00494724526289847 ] ]
Measured Statistics: {meanExponent=-2.6040974946745723, negative=9, min=-0.0053114596376691026, max=0.009682032139496366, mean=1.5612511283791264E-13, count=16, sum=2.4980018054066022E-12, positive=7, stdDev=0.004517373337162912, zeros=0}
Feedback Error: [ [ -7.059109595475288E-9, 4.626799326480635E-9, -8.460028616237916E-10, 3.2783131300763435E-9 ], [ 5.287691356818669E-10, -2.9606630706086046E-9, -8.460028616237916E-10, 3.2783131300763435E-9 ], [ 5.28907913559945E-10, 4.62790954950526E-9, -8.434020369249562E-9, 3.278868241588656E-9 ], [ 5.287691356818669E-10, 4.626799326480635E-9, -8.460028616237916E-10, -4.309149266579215E-9 ] ]
Error Statistics: {meanExponent=-8.664027258005207, negative=7, min=-8.434020369249562E-9, max=4.62790954950526E-9, mean=1.5612514671923053E-13, count=16, sum=2.4980023475076885E-12, positive=9, stdDev=3.935113261714419E-9, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 16,
        "sum" : 5.06043997759156E-8,
        "min" : 5.287691356818669E-10,
        "max" : 8.434020369249562E-9,
        "sumOfSquare" : 2.477618625103322E-16,
        "standardDeviation" : 2.34136088522505E-9,
        "average" : 3.162774985994725E-9
      },
      "relativeTol" : {
        "count" : 16,
        "sum" : 6.968832673971972E-6,
        "min" : 4.3547671903803336E-7,
        "max" : 4.3565634911656297E-7,
        "sumOfSquare" : 3.035289336748485E-12,
        "standardDeviation" : 4.635783080739171E-11,
        "average" : 4.3555204212324827E-7
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 16,
        "sum" : 5.06043997759156E-8,
        "min" : 5.287691356818669E-10,
        "max" : 8.434020369249562E-9,
        "sumOfSquare" : 2.477618625103322E-16,
        "standardDeviation" : 2.34136088522505E-9,
        "average" : 3.162774985994725E-9
      },
      "relativeTol" : {
        "count" : 16,
        "sum" : 6.968832673971972E-6,
        "min" : 4.3547671903803336E-7,
        "max" : 4.3565634911656297E-7,
        "sumOfSquare" : 3.035289336748485E-12,
        "standardDeviation" : 4.635783080739171E-11,
        "average" : 4.3555204212324827E-7
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 3.1628e-09 +- 2.3414e-09 [5.2877e-10 - 8.4340e-09] (16#)
relativeTol: 4.3555e-07 +- 4.6358e-11 [4.3548e-07 - 4.3566e-07] (16#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=3.1628e-09 +- 2.3414e-09 [5.2877e-10 - 8.4340e-09] (16#), relativeTol=4.3555e-07 +- 4.6358e-11 [4.3548e-07 - 4.3566e-07] (16#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.167",
      "gc_time": "0.110"
    },
    "created_on": 1586738552534,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.L1NormalizationLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/L1NormalizationLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/L1NormalizationLayer/Basic/derivativeTest/202004134232",
    "id": "d76d1e75-c79e-4073-a240-91e4c891b3be",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "L1NormalizationLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.L1NormalizationLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/L1NormalizationLayer.java",
      "javaDoc": ""
    }
  }