1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 2770482254418328576

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [ 0.08, 0.7, -0.128, 0.496, -0.608 ]
Inputs Statistics: {meanExponent=-0.5330433495165998, negative=2, min=-0.608, max=0.7, mean=0.10799999999999998, count=5, sum=0.5399999999999999, positive=3, stdDev=0.4626324675160618, zeros=0}
Output: [ 0.1757592438713306, 0.3267237870461263, 0.14275289463946447, 0.2664309541079262, 0.0883331203351525 ]
Outputs Statistics: {meanExponent=-0.7429215789557428, negative=0, min=0.0883331203351525, max=0.3267237870461263, mean=0.20000000000000004, count=5, sum=1.0000000000000002, positive=5, stdDev=0.08579781723439674, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.03 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [ 0.08, 0.7, -0.128, 0.496, -0.608 ]
Value Statistics: {meanExponent=-0.5330433495165998, negative=2, min=-0.608, max=0.7, mean=0.10799999999999998, count=5, sum=0.5399999999999999, positive=3, stdDev=0.4626324675160618, zeros=0}
Implemented Feedback: [ [ 0.14486793206510873, -0.057424725766004804, -0.025090140822276005, -0.046827703037926306, -0.01552536243890166 ], [ -0.057424725766004804, 0.21997535402436383, -0.0466407663484025, -0.08704933031245435, -0.028860531597502216 ], [ -0.025090140822276005, -0.0466407663484025, 0.12237450571151842, -0.03803378992046079, -0.012609808620379163 ], [ -0.046827703037926306, -0.08704933031245435, -0.03803378992046079, 0.19544550080106635, -0.023534677530224944 ], [ -0.01552536243890166, -0.028860531597502216, -0.012609808620379163, -0.023534677530224944, 0.08053038018700798 ] ]
Implemented Statistics: {meanExponent=-1.3571536197901788, negative=20, min=-0.08704933031245435, max=0.21997535402436383, mean=-6.661338147750939E-18, count=25, sum=-1.6653345369377348E-16, positive=5, stdDev=0.08179196027059633, zeros=0}
Measured Feedback: [ [ 0.14487262930518163, -0.05742658772289744, -0.025090954352546824, -0.0468292213934296, -0.015525865837973107 ], [ -0.057425720769443966, 0.21997916555605013, -0.04664157449735251, -0.08705083862414753, -0.02886103166691023 ], [ -0.025091037171576236, -0.04664243259733247, 0.12237887755950361, -0.03803514868339075, -0.012610259107898036 ], [ -0.04682879677475604, -0.0870513634909642, -0.03803467826135609, 0.19545006574639423, -0.023535227220844446 ], [ -0.01552600158022388, -0.02886171971527851, -0.012610327735362858, -0.023535646395123067, 0.08053369542501687 ] ]
Measured Statistics: {meanExponent=-1.3571405911039411, negative=20, min=-0.0870513634909642, max=0.21997916555605013, mean=-2.6645352591003757E-13, count=25, sum=-6.661338147750939E-12, positive=5, stdDev=0.08179399761931282, zeros=0}
Feedback Error: [ [ 4.697240072903952E-6, -1.8619568926334096E-6, -8.135302708191239E-7, -1.5183555032924345E-6, -5.033990714466835E-7 ], [ -9.95003439162312E-7, 3.8115316862985527E-6, -8.081489500100236E-7, -1.5083116931824403E-6, -5.000694080145585E-7 ], [ -8.963493002313305E-7, -1.666248929971248E-6, 4.3718479851850844E-6, -1.3587629299621762E-6, -4.5048751887329097E-7 ], [ -1.0937368297367356E-6, -2.03317850985818E-6, -8.883408953025418E-7, 4.564945327883985E-6, -5.496906195015527E-7 ], [ -6.391413222197467E-7, -1.1881177762948503E-6, -5.191149836953485E-7, -9.68864898122368E-7, 3.315238008894106E-6 ] ]
Error Statistics: {meanExponent=-5.900046505572036, negative=20, min=-2.03317850985818E-6, max=4.697240072903952E-6, mean=-2.6644658701613366E-13, count=25, sum=-6.6611646754033416E-12, positive=5, stdDev=2.1304352017223066E-6, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 25,
        "sum" : 4.1521612823496035E-5,
        "min" : 4.5048751887329097E-7,
        "max" : 4.697240072903952E-6,
        "sumOfSquare" : 1.1346885371844088E-10,
        "standardDeviation" : 1.3342725427718054E-6,
        "average" : 1.6608645129398414E-6
      },
      "relativeTol" : {
        "count" : 25,
        "sum" : 3.7499573386529106E-4,
        "min" : 8.663467510081774E-6,
        "max" : 2.0583358224487338E-5,
        "sumOfSquare" : 6.084977822076782E-9,
        "standardDeviation" : 4.29001540971617E-6,
        "average" : 1.4999829354611642E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 25,
        "sum" : 4.1521612823496035E-5,
        "min" : 4.5048751887329097E-7,
        "max" : 4.697240072903952E-6,
        "sumOfSquare" : 1.1346885371844088E-10,
        "standardDeviation" : 1.3342725427718054E-6,
        "average" : 1.6608645129398414E-6
      },
      "relativeTol" : {
        "count" : 25,
        "sum" : 3.7499573386529106E-4,
        "min" : 8.663467510081774E-6,
        "max" : 2.0583358224487338E-5,
        "sumOfSquare" : 6.084977822076782E-9,
        "standardDeviation" : 4.29001540971617E-6,
        "average" : 1.4999829354611642E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 1.6609e-06 +- 1.3343e-06 [4.5049e-07 - 4.6972e-06] (25#)
relativeTol: 1.5000e-05 +- 4.2900e-06 [8.6635e-06 - 2.0583e-05] (25#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.6609e-06 +- 1.3343e-06 [4.5049e-07 - 4.6972e-06] (25#), relativeTol=1.5000e-05 +- 4.2900e-06 [8.6635e-06 - 2.0583e-05] (25#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.177",
      "gc_time": "0.106"
    },
    "created_on": 1586739516990,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "SubBatchLayerTest",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.SubBatchLayerTest",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/test/java/com/simiacryptus/mindseye/layers/java/SubBatchLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/java/SubBatchLayer/SubBatchLayerTest/derivativeTest/202004135836",
    "id": "be4e6e68-e27a-4e49-91f9-56b4cd0d6699",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "SubBatchLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.java.SubBatchLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-java/tree/93db34cedee48c0202777a2b25deddf1dfaf5731/src/main/java/com/simiacryptus/mindseye/layers/java/SubBatchLayer.java",
      "javaDoc": ""
    }
  }