1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 6501802227268311040

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Inputs Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Output: [
[ [ 0.21776758783465985, 0.1094456832971881, 0.672786728868152 ], [ 0.38476222787786835, 0.4588051240527918, 0.15643264806933976 ] ],
[ [ 0.2508238804179965, 0.7268724544377955, 0.02230366514420795 ], [ 0.23751546211083552, 0.6639671829454358, 0.09851735494372868 ] ]
]
Outputs Statistics: {meanExponent=-0.6294356703587121, negative=0, min=0.02230366514420795, max=0.7268724544377955, mean=0.3333333333333333, count=12, sum=4.0, positive=12, stdDev=0.23460078463826506, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.08 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Value Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Implemented Feedback: [ [ 0.17034486552333358, 0.0, 0.0, 0.0, -0.023833722450544777, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.1879112614300551, 0.0, 0.0, 0.0, -0.1823169695910412, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.23672025587632767, 0.0, 0.0, 0.0, -0.17653088169233394, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.18110186736911174, 0.0, 0.0, 0.0, -0.15770247228371484, ... ], [ -0.023833722450544777, 0.0, 0.0, 0.0, 0.0974673257047997, 0.0, 0.0, 0.0, ... ], [ 0.0, -0.1823169695910412, 0.0, 0.0, 0.0, 0.19852888941737043, 0.0, 0.0, ... ], [ 0.0, 0.0, -0.17653088169233394, 0.0, 0.0, 0.0, 0.24830298219569413, 0.0, ... ], [ 0.0, 0.0, 0.0, -0.15770247228371484, 0.0, 0.0, 0.0, 0.22311476291693796, ... ], ... ]
Implemented Statistics: {meanExponent=-1.119307587299284, negative=24, min=-0.1823169695910412, max=0.24830298219569413, mean=1.927470528863119E-18, count=144, sum=2.7755575615628914E-16, positive=12, stdDev=0.06713358813205941, zeros=108}
Measured Feedback: [ [ 0.1703496732013976, 0.0, 0.0, 0.0, -0.02383439511449703, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.1879159436901645, 0.0, 0.0, 0.0, -0.1823215124552302, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.23672298362198418, 0.0, 0.0, 0.0, -0.17653291587094166, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.1811066209869705, 0.0, 0.0, 0.0, -0.15770661170644296, ... ], [ -0.02383465330324519, 0.0, 0.0, 0.0, 0.09747113240046734, 0.0, 0.0, 0.0, ... ], [ 0.0, -0.18231283326342052, 0.0, 0.0, 0.0, 0.198524385280896, 0.0, 0.0, ... ], [ 0.0, 0.0, -0.17653160876540674, 0.0, 0.0, 0.0, 0.2483040048745222, 0.0, ... ], [ 0.0, 0.0, 0.0, -0.1576998863916934, 0.0, 0.0, 0.0, 0.22311110444150906, ... ], ... ]
Measured Statistics: {meanExponent=-1.11930034924067, negative=24, min=-0.1823215124552302, max=0.2483040048745222, mean=2.6020852139652106E-14, count=144, sum=3.747002708109903E-12, positive=12, stdDev=0.0671339945922998, zeros=108}
Feedback Error: [ [ 4.807678064017162E-6, 0.0, 0.0, 0.0, -6.726639522544597E-7, 0.0, 0.0, 0.0, ... ], [ 0.0, 4.682260109412084E-6, 0.0, 0.0, 0.0, -4.542864188994411E-6, 0.0, 0.0, ... ], [ 0.0, 0.0, 2.727745656516678E-6, 0.0, 0.0, 0.0, -2.0341786077238133E-6, 0.0, ... ], [ 0.0, 0.0, 0.0, 4.753617858760384E-6, 0.0, 0.0, 0.0, -4.139422728116626E-6, ... ], [ -9.308527004134315E-7, 0.0, 0.0, 0.0, 3.806695667643578E-6, 0.0, 0.0, 0.0, ... ], [ 0.0, 4.136327620696267E-6, 0.0, 0.0, 0.0, -4.504136474431375E-6, 0.0, 0.0, ... ], [ 0.0, 0.0, -7.2707307280373E-7, 0.0, 0.0, 0.0, 1.0226788280665033E-6, 0.0, ... ], [ 0.0, 0.0, 0.0, 2.5858920214283376E-6, 0.0, 0.0, 0.0, -3.658475428908181E-6, ... ], ... ]
Error Statistics: {meanExponent=-5.772543829667077, negative=21, min=-4.542864188994411E-6, max=4.807678064017162E-6, mean=2.6019105369485323E-14, count=144, sum=3.746751173205887E-12, positive=15, stdDev=1.431581345602887E-6, zeros=108}

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 8.581687049558952E-5,
        "min" : 0.0,
        "max" : 4.807678064017162E-6,
        "sumOfSquare" : 2.9511722146725704E-10,
        "standardDeviation" : 1.3016405660089368E-6,
        "average" : 5.959504895527049E-7
      },
      "relativeTol" : {
        "count" : 36,
        "sum" : 4.6908620643034516E-4,
        "min" : 2.059331860715274E-6,
        "max" : 2.3884969562578075E-5,
        "sumOfSquare" : 7.453358843433546E-9,
        "standardDeviation" : 6.103470558235146E-6,
        "average" : 1.303017240084292E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 8.581687049558952E-5,
        "min" : 0.0,
        "max" : 4.807678064017162E-6,
        "sumOfSquare" : 2.9511722146725704E-10,
        "standardDeviation" : 1.3016405660089368E-6,
        "average" : 5.959504895527049E-7
      },
      "relativeTol" : {
        "count" : 36,
        "sum" : 4.6908620643034516E-4,
        "min" : 2.059331860715274E-6,
        "max" : 2.3884969562578075E-5,
        "sumOfSquare" : 7.453358843433546E-9,
        "standardDeviation" : 6.103470558235146E-6,
        "average" : 1.303017240084292E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 5.9595e-07 +- 1.3016e-06 [0.0000e+00 - 4.8077e-06] (144#)
relativeTol: 1.3030e-05 +- 6.1035e-06 [2.0593e-06 - 2.3885e-05] (36#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=5.9595e-07 +- 1.3016e-06 [0.0000e+00 - 4.8077e-06] (144#), relativeTol=1.3030e-05 +- 6.1035e-06 [2.0593e-06 - 2.3885e-05] (36#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.214",
      "gc_time": "0.098"
    },
    "created_on": 1586735136190,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgPixelSoftmaxLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayer/Basic/derivativeTest/202004124536",
    "id": "83d11b6e-9d76-4b29-a6fe-b3942296e0d9",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ImgPixelSoftmaxLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.ImgPixelSoftmaxLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/ImgPixelSoftmaxLayer.java",
      "javaDoc": ""
    }
  }