1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging 
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine4019855894743320889/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 3507240607550044160

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -1.72, -0.712, -0.176 ], [ 0.496, -0.852, 1.356, -0.768 ], [ 0.048, -0.804, 1.032, 1.64 ] ],
[ [ 0.7, -1.028, 1.048, 1.556 ], [ -0.608, 1.912, 1.512, -0.068 ], [ 1.524, 1.108, 0.3, 0.392 ] ],
[ [ -0.128, -0.384, -1.616, 0.788 ], [ 1.764, -1.688, 1.556, 1.612 ], [ 1.208, 0.028, 0.636, 0.092 ] ]
]
Inputs Statistics: {meanExponent=-0.2192709808999687, negative=13, min=-1.72, max=1.912, mean=0.3287777777777778, count=36, sum=11.836, positive=23, stdDev=1.0387145129948714, zeros=0}
Output: [
[ [ 2.49576, -6.9094560000000005, 0.7270559999999981, 2.0843360000000004 ], [ 0.9096640000000011, 1.0257119999999997, -8.954239999999997, -7.821728000000001 ], [ -4.891376000000002, -3.267375999999999, -7.408304000000001, -11.039343999999998 ] ],
[ [ -0.924112, 4.012847999999999, -8.408384, -15.7688 ], [ 10.398096, -11.811296000000002, 6.691167999999999, -8.880287999999998 ], [ 2.76104, 0.37903999999999977, 1.7782719999999996, -1.691232 ] ],
[ [ -2.7663840000000004, -6.1763840000000005, -5.756912000000001, 1.7154879999999997 ], [ 11.102752, -4.398512, -2.1475200000000005, -4.243984000000001 ], [ 3.8906560000000003, -5.078335999999998, 0.8235359999999997, 0.21856000000000048 ] ]
]
Outputs Statistics: {meanExponent=0.523005813869163, negative=20, min=-15.7688, max=11.102752, mean=-2.1480551111111112, count=36, sum=-77.32998400000001, positive=16, stdDev=5.9101195418406895, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 1.27 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -1.72, -0.712, -0.176 ], [ 0.496, -0.852, 1.356, -0.768 ], [ 0.048, -0.804, 1.032, 1.64 ] ],
[ [ 0.7, -1.028, 1.048, 1.556 ], [ -0.608, 1.912, 1.512, -0.068 ], [ 1.524, 1.108, 0.3, 0.392 ] ],
[ [ -0.128, -0.384, -1.616, 0.788 ], [ 1.764, -1.688, 1.556, 1.612 ], [ 1.208, 0.028, 0.636, 0.092 ] ]
]
Value Statistics: {meanExponent=-0.2192709808999687, negative=13, min=-1.72, max=1.912, mean=0.3287777777777778, count=36, sum=11.836, positive=23, stdDev=1.0387145129948714, zeros=0}
Implemented Feedback: [ [ -0.312, -1.256, 0.0, -1.1, -0.784, 0.0, 0.0, 0.0, ... ], [ -1.58, -0.312, -1.256, 0.012, -1.1, -0.784, 0.0, 0.0, ... ], [ 0.0, -1.58, -0.312, 0.0, 0.012, -1.1, 0.0, 0.0, ... ], [ 1.972, 1.24, 0.0, -0.312, -1.256, 0.0, -1.1, -0.784, ... ], [ 0.672, 1.972, 1.24, -1.58, -0.312, -1.256, 0.012, -1.1, ... ], [ 0.0, 0.672, 1.972, 0.0, -1.58, -0.312, 0.0, 0.012, ... ], [ 0.0, 0.0, 0.0, 1.972, 1.24, 0.0, -0.312, -1.256, ... ], [ 0.0, 0.0, 0.0, 0.672, 1.972, 1.24, -1.58, -0.312, ... ], ... ]
Implemented Statistics: {meanExponent=-0.11690546046195874, negative=412, min=-2.0, max=1.98, mean=-0.018839506172839714, count=1296, sum=-24.41600000000027, positive=372, stdDev=0.9338546884713927, zeros=512}
Measured Feedback: [ [ -0.31200000000009, -1.25600000000059, 0.0, -1.0999999999983245, -0.7840000000136627, 0.0, 0.0, 0.0, ... ], [ -1.5799999999988046, -0.31200000000009, -1.25600000000059, 0.011999999998124622, -1.1000000000116472, -0.7839999999958991, 0.0, 0.0, ... ], [ 0.0, -1.5800000000032455, -0.31200000000009, 0.0, 0.011999999998124622, -1.0999999999938836, 0.0, 0.0, ... ], [ 1.972000000001195, 1.2400000000001299, 0.0, -0.31200000000009, -1.2560000000050309, 0.0, -1.1000000000027654, -0.78400000000034, ... ], [ 0.6720000000015602, 1.9719999999967541, 1.2400000000001299, -1.5800000000032455, -0.3120000000045309, -1.2560000000050309, 0.012000000007006406, -1.1000000000027654, ... ], [ 0.0, 0.6720000000015602, 1.972000000001195, 0.0, -1.5800000000076864, -0.3120000000045309, 0.0, 0.012000000002565514, ... ], [ 0.0, 0.0, 0.0, 1.9719999999967541, 1.239999999995689, 0.0, -0.3119999999956491, -1.25600000000059, ... ], [ 0.0, 0.0, 0.0, 0.6719999999971193, 1.9719999999878723, 1.239999999995689, -1.5799999999988046, -0.31200000000009, ... ], ... ]
Measured Statistics: {meanExponent=-0.11690546046100435, negative=412, min=-2.0000000000042206, max=1.9800000000103068, mean=-0.018839506173213304, count=1296, sum=-24.41600000048444, positive=372, stdDev=0.9338546884711789, zeros=512}
Feedback Error: [ [ -8.998357614586894E-14, -5.899725152858082E-13, 0.0, 1.6755485887642863E-12, -1.3662626585642101E-11, 0.0, 0.0, 0.0, ... ], [ 1.1954881529163686E-12, -8.998357614586894E-14, -5.899725152858082E-13, -1.8753783248559586E-12, -1.1647127706737592E-11, 4.100941808360403E-12, 0.0, 0.0, ... ], [ 0.0, -3.2454039455842576E-12, -8.998357614586894E-14, 0.0, -1.8753783248559586E-12, 6.1164406872649124E-12, 0.0, 0.0, ... ], [ 1.1950440637065185E-12, 1.2989609388114332E-13, 0.0, -8.998357614586894E-14, -5.030864613786434E-12, 0.0, -2.76534350973634E-12, -3.3995029014022293E-13, ... ], [ 1.5601964165057325E-12, -3.2458480347941077E-12, 1.2989609388114332E-13, -3.2454039455842576E-12, -4.530875674646495E-12, -5.030864613786434E-12, 7.006405872145294E-12, -2.76534350973634E-12, ... ], [ 0.0, 1.5601964165057325E-12, 1.1950440637065185E-12, 0.0, -7.686296044084884E-12, -4.530875674646495E-12, 0.0, 2.5655137736446676E-12, ... ], [ 0.0, 0.0, 0.0, -3.2458480347941077E-12, -4.310996004619483E-12, 0.0, 4.350908522354757E-12, -5.899725152858082E-13, ... ], [ 0.0, 0.0, 0.0, -2.8806956819948937E-12, -1.212763223179536E-11, -4.310996004619483E-12, 1.1954881529163686E-12, -8.998357614586894E-14, ... ], ... ]
Error Statistics: {meanExponent=-11.74316533939359, negative=425, min=-1.6133316904642925E-11, max=1.4818146709671964E-11, mean=-3.7383475355521636E-13, count=1296, sum=-4.844898406075604E-10, positive=359, stdDev=3.755536628235949E-12, zeros=512}

Returns

    {
      "absoluteTol" : {
        "count" : 1296,
        "sum" : 2.6966556869456593E-9,
        "min" : 0.0,
        "max" : 1.6133316904642925E-11,
        "sumOfSquare" : 1.845997489452784E-20,
        "standardDeviation" : 3.148694235325455E-12,
        "average" : 2.080752844865478E-12
      },
      "relativeTol" : {
        "count" : 784,
        "sum" : 5.4840305680258276E-9,
        "min" : 6.251255769285792E-17,
        "max" : 2.9193357792082866E-10,
        "sumOfSquare" : 6.371971099730128E-19,
        "standardDeviation" : 2.7637334764069478E-11,
        "average" : 6.994936949012535E-12
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.96 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Learning Gradient for weight setByCoord 0
Weights: [ 0.672, 1.972, 1.24, -1.58, -0.312, -1.256, 0.012, -1.1, ... ]
Implemented Gradient: [ [ -0.608, 1.764, 0.0, 1.524, 1.208, 0.0, 0.0, 0.0, ... ], [ 0.496, -0.608, 1.764, 0.048, 1.524, 1.208, 0.0, 0.0, ... ], [ 0.0, 0.496, -0.608, 0.0, 0.048, 1.524, 0.0, 0.0, ... ], [ 0.7, -0.128, 0.0, -0.608, 1.764, 0.0, 1.524, 1.208, ... ], [ 0.08, 0.7, -0.128, 0.496, -0.608, 1.764, 0.048, 1.524, ... ], [ 0.0, 0.08, 0.7, 0.0, 0.496, -0.608, 0.0, 0.048, ... ], [ 0.0, 0.0, 0.0, 0.7, -0.128, 0.0, -0.608, 1.764, ... ], [ 0.0, 0.0, 0.0, 0.08, 0.7, -0.128, 0.496, -0.608, ... ], ... ]
Implemented Statistics: {meanExponent=-0.185813247795393, negative=280, min=-1.72, max=1.912, mean=0.061138888888889055, count=5184, sum=316.94400000000087, positive=504, stdDev=0.4340364656908257, zeros=4400}
Measured Gradient: [ [ -0.6079999999997199, 1.7639999999996547, 0.0, 1.5239999999971943, 1.2079999999947688, 0.0, 0.0, 0.0, ... ], [ 0.49600000000094013, -0.6079999999997199, 1.7639999999996547, 0.04799999999693938, 1.5239999999927534, 1.2079999999947688, 0.0, 0.0, ... ], [ 0.0, 0.49600000000094013, -0.6079999999997199, 0.0, 0.04799999999249849, 1.5239999999927534, 0.0, 0.0, ... ], [ 0.700000000000145, -0.12799999999923983, 0.0, -0.6079999999997199, 1.763999999990773, 0.0, 1.5240000000016352, 1.2079999999992097, ... ], [ 0.08000000000230045, 0.700000000000145, -0.12799999999923983, 0.49600000000094013, -0.6080000000174834, 1.763999999990773, 0.04800000000138027, 1.5239999999971943, ... ], [ 0.0, 0.07999999999785956, 0.700000000000145, 0.0, 0.49599999998761746, -0.6079999999997199, 0.0, 0.04799999999693938, ... ], [ 0.0, 0.0, 0.0, 0.700000000000145, -0.12800000000368073, 0.0, -0.6079999999997199, 1.7639999999996547, ... ], [ 0.0, 0.0, 0.0, 0.07999999999785956, 0.6999999999912632, -0.12800000000368073, 0.49599999999649924, -0.6079999999997199, ... ], ... ]
Measured Statistics: {meanExponent=-0.1858132477953767, negative=280, min=-1.7200000000094917, max=1.9120000000150128, mean=0.06113888888880951, count=5184, sum=316.94399999958847, positive=504, stdDev=0.4340364656907308, zeros=4400}
Gradient Error: [ [ 2.80109269112927E-13, -3.452793606584237E-13, 0.0, -2.8057556278326956E-12, -5.2311488474288126E-12, 0.0, 0.0, 0.0, ... ], [ 9.401368572525826E-13, 2.80109269112927E-13, -3.452793606584237E-13, -3.060621200923208E-12, -7.246647726333322E-12, -5.2311488474288126E-12, 0.0, 0.0, ... ], [ 0.0, 9.401368572525826E-13, 2.80109269112927E-13, 0.0, -7.501513299423834E-12, -7.246647726333322E-12, 0.0, 0.0, ... ], [ 1.4499512701604544E-13, 7.601697049608447E-13, 0.0, 2.80109269112927E-13, -9.227063557659676E-12, 0.0, 1.6351364706679306E-12, -7.902567489281864E-13, ... ], [ 2.3004514959623634E-12, 1.4499512701604544E-13, 7.601697049608447E-13, 9.401368572525826E-13, -1.7483459124889578E-11, -9.227063557659676E-12, 1.380270897577418E-12, -2.8057556278326956E-12, ... ], [ 0.0, -2.1404406025382627E-12, 1.4499512701604544E-13, 0.0, -1.2382539438249296E-11, 2.80109269112927E-13, 0.0, -3.060621200923208E-12, ... ], [ 0.0, 0.0, 0.0, 1.4499512701604544E-13, -3.6807223935397815E-12, 0.0, 2.80109269112927E-13, -3.452793606584237E-13, ... ], [ 0.0, 0.0, 0.0, -2.1404406025382627E-12, -8.736789069985207E-12, -3.6807223935397815E-12, -3.5007552412480436E-12, 2.80109269112927E-13, ... ], ... ]
Error Statistics: {meanExponent=-11.729581392405505, negative=430, min=-1.7483459124889578E-11, max=1.5713097489822303E-11, mean=-7.938022881333518E-14, count=5184, sum=-4.1150710616832953E-10, positive=354, stdDev=1.9385316531010254E-12, zeros=4400}

Returns

    {
      "absoluteTol" : {
        "count" : 6480,
        "sum" : 5.487513814877998E-9,
        "min" : 0.0,
        "max" : 1.7483459124889578E-11,
        "sumOfSquare" : 3.797361978764049E-20,
        "standardDeviation" : 2.267816345701094E-12,
        "average" : 8.468385516787034E-13
      },
      "relativeTol" : {
        "count" : 1568,
        "sum" : 1.0977686450563385E-8,
        "min" : 6.251255769285792E-17,
        "max" : 2.9193357792082866E-10,
        "sumOfSquare" : 9.490886847410045E-19,
        "standardDeviation" : 2.3585399980319942E-11,
        "average" : 7.001075542451139E-12
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 8.4684e-13 +- 2.2678e-12 [0.0000e+00 - 1.7483e-11] (6480#)
relativeTol: 7.0011e-12 +- 2.3585e-11 [6.2513e-17 - 2.9193e-10] (1568#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.02 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=8.4684e-13 +- 2.2678e-12 [0.0000e+00 - 1.7483e-11] (6480#), relativeTol=7.0011e-12 +- 2.3585e-11 [6.2513e-17 - 2.9193e-10] (1568#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "2.882",
      "gc_time": "0.455"
    },
    "created_on": 1586746719708,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Double",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.Double",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/Double/derivativeTest/202004135839",
    "id": "d1b1937e-ac10-4284-a5cd-509b3fc3c2e8",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ConvolutionLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
      "javaDoc": ""
    }
  }