1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 8708343567618667520

Differential Validation

SingleDerivativeTester.java:101 executed in 0.01 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [
[ [ 0.08, -0.472 ], [ 1.208, -0.316 ], [ 1.108, -1.552 ], [ 1.032, -1.484 ], [ 1.612, -1.248 ], [ 1.552, 0.56 ], [ -0.804, -0.312 ], [ -1.832, 0.812 ] ],
[ [ 0.7, -0.504 ], [ -1.72, 1.156 ], [ 0.028, 0.016 ], [ 0.3, 1.352 ], [ 1.64, 1.324 ], [ 1.876, -1.656 ], [ 0.148, -0.968 ], [ 1.368, -1.808 ] ],
[ [ -0.128, -1.156 ], [ -1.028, 0.972 ], [ -0.712, 1.288 ], [ 0.636, -0.888 ], [ 0.392, 0.344 ], [ -0.408, -0.856 ], [ -0.032, -0.892 ], [ -1.54, 0.644 ] ],
[ [ 0.496, 0.184 ], [ -0.384, -1.116 ], [ 1.048, 1.628 ], [ -0.176, -1.256 ], [ 0.092, -1.34 ], [ -0.384, -1.16 ], [ -0.892, -0.808 ], [ -0.876, 0.66 ] ],
[ [ -0.608, 1.98 ], [ -0.852, 0.52 ], [ -1.616, -1.564 ], [ 1.556, 1.916 ], [ -0.556, 1.776 ], [ -1.572, 0.688 ], [ 1.62, -1.1 ], [ 1.652, -1.76 ] ],
[ [ 1.764, -0.628 ], [ 1.912, -1.456 ], [ 1.356, -1.764 ], [ 0.788, -1.724 ], [ -1.476, 0.52 ], [ -1.516, -0.124 ], [ -1.856, -0.968 ], [ -1.424, 0.82 ] ],
[ [ 0.048, -0.368 ], [ -1.688, -2.0 ], [ 1.512, 0.692 ], [ -0.768, -1.664 ], [ 1.704, 1.444 ], [ -0.636, -1.176 ], [ 0.996, -0.784 ], [ -0.464, 1.24 ] ],
[ [ 1.524, -1.16 ], [ -0.804, 1.42 ], [ 1.556, 1.628 ], [ -0.068, 0.82 ], [ -1.228, 1.956 ], [ -1.492, -1.256 ], [ 0.048, 0.012 ], [ -0.012, -1.58 ] ]
]
Inputs Statistics: {meanExponent=-0.13117442306991567, negative=68, min=-2.0, max=1.98, mean=-0.06793749999999998, count=128, sum=-8.695999999999998, positive=60, stdDev=1.1762083132225136, zeros=0}
Output: [
[ [ 0.636, -0.888 ], [ 0.392, 0.344 ] ],
[ [ -0.176, -1.256 ], [ 0.092, -1.34 ] ],
[ [ 1.556, 1.916 ], [ -0.556, 1.776 ] ]
]
Outputs Statistics: {meanExponent=-0.184497302136206, negative=5, min=-1.34, max=1.916, mean=0.208, count=12, sum=2.496, positive=7, stdDev=1.076798959880627, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.13 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [
[ [ 0.08, -0.472 ], [ 1.208, -0.316 ], [ 1.108, -1.552 ], [ 1.032, -1.484 ], [ 1.612, -1.248 ], [ 1.552, 0.56 ], [ -0.804, -0.312 ], [ -1.832, 0.812 ] ],
[ [ 0.7, -0.504 ], [ -1.72, 1.156 ], [ 0.028, 0.016 ], [ 0.3, 1.352 ], [ 1.64, 1.324 ], [ 1.876, -1.656 ], [ 0.148, -0.968 ], [ 1.368, -1.808 ] ],
[ [ -0.128, -1.156 ], [ -1.028, 0.972 ], [ -0.712, 1.288 ], [ 0.636, -0.888 ], [ 0.392, 0.344 ], [ -0.408, -0.856 ], [ -0.032, -0.892 ], [ -1.54, 0.644 ] ],
[ [ 0.496, 0.184 ], [ -0.384, -1.116 ], [ 1.048, 1.628 ], [ -0.176, -1.256 ], [ 0.092, -1.34 ], [ -0.384, -1.16 ], [ -0.892, -0.808 ], [ -0.876, 0.66 ] ],
[ [ -0.608, 1.98 ], [ -0.852, 0.52 ], [ -1.616, -1.564 ], [ 1.556, 1.916 ], [ -0.556, 1.776 ], [ -1.572, 0.688 ], [ 1.62, -1.1 ], [ 1.652, -1.76 ] ],
[ [ 1.764, -0.628 ], [ 1.912, -1.456 ], [ 1.356, -1.764 ], [ 0.788, -1.724 ], [ -1.476, 0.52 ], [ -1.516, -0.124 ], [ -1.856, -0.968 ], [ -1.424, 0.82 ] ],
[ [ 0.048, -0.368 ], [ -1.688, -2.0 ], [ 1.512, 0.692 ], [ -0.768, -1.664 ], [ 1.704, 1.444 ], [ -0.636, -1.176 ], [ 0.996, -0.784 ], [ -0.464, 1.24 ] ],
[ [ 1.524, -1.16 ], [ -0.804, 1.42 ], [ 1.556, 1.628 ], [ -0.068, 0.82 ], [ -1.228, 1.956 ], [ -1.492, -1.256 ], [ 0.048, 0.012 ], [ -0.012, -1.58 ] ]
]
Value Statistics: {meanExponent=-0.13117442306991567, negative=68, min=-2.0, max=1.98, mean=-0.06793749999999998, count=128, sum=-8.695999999999998, positive=60, stdDev=1.1762083132225136, zeros=0}
Implemented Feedback: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
Implemented Statistics: {meanExponent=0.0, negative=0, min=0.0, max=1.0, mean=0.0078125, count=1536, sum=12.0, positive=12, stdDev=0.08804240366863003, zeros=1524}
Measured Feedback: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
Measured Statistics: {meanExponent=-4.28081034527471E-14, negative=0, min=0.0, max=1.0000000000000286, mean=0.00781249999999923, count=1536, sum=11.999999999998817, positive=12, stdDev=0.08804240366862136, zeros=1524}
Feedback Error: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
Error Statistics: {meanExponent=-13.006819095219404, negative=11, min=-1.1013412404281553E-13, max=2.864375403532904E-14, mean=-7.700726630440376E-16, count=1536, sum=-1.1828316104356418E-12, positive=1, stdDev=9.316986350214105E-15, zeros=1524}

Returns

    {
      "absoluteTol" : {
        "count" : 1536,
        "sum" : 1.2401191185062999E-12,
        "min" : 0.0,
        "max" : 1.1013412404281553E-13,
        "sumOfSquare" : 1.3424524271069748E-25,
        "standardDeviation" : 9.313828509406266E-15,
        "average" : 8.07369217777539E-16
      },
      "relativeTol" : {
        "count" : 12,
        "sum" : 6.200595592531831E-13,
        "min" : 1.4321877017664317E-14,
        "max" : 5.50670620214108E-14,
        "sumOfSquare" : 3.3561310677678044E-26,
        "standardDeviation" : 1.1261374222586905E-14,
        "average" : 5.167162993776526E-14
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 1536,
        "sum" : 1.2401191185062999E-12,
        "min" : 0.0,
        "max" : 1.1013412404281553E-13,
        "sumOfSquare" : 1.3424524271069748E-25,
        "standardDeviation" : 9.313828509406266E-15,
        "average" : 8.07369217777539E-16
      },
      "relativeTol" : {
        "count" : 12,
        "sum" : 6.200595592531831E-13,
        "min" : 1.4321877017664317E-14,
        "max" : 5.50670620214108E-14,
        "sumOfSquare" : 3.3561310677678044E-26,
        "standardDeviation" : 1.1261374222586905E-14,
        "average" : 5.167162993776526E-14
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 8.0737e-16 +- 9.3138e-15 [0.0000e+00 - 1.1013e-13] (1536#)
relativeTol: 5.1672e-14 +- 1.1261e-14 [1.4322e-14 - 5.5067e-14] (12#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=8.0737e-16 +- 9.3138e-15 [0.0000e+00 - 1.1013e-13] (1536#), relativeTol=5.1672e-14 +- 1.1261e-14 [1.4322e-14 - 5.5067e-14] (12#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.283",
      "gc_time": "0.104"
    },
    "created_on": 1586735547460,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgViewLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/ImgViewLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/ImgViewLayer/Basic/derivativeTest/202004125227",
    "id": "618a4e8a-d4ec-4fa9-a3ad-1779bbe39d78",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ImgViewLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgViewLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/ImgViewLayer.java",
      "javaDoc": ""
    }
  }