1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine3395840269325606830/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 1450504107269727232

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556, -1.516, 0.996 ], [ 0.496, -0.852, 1.356, -0.768, -1.228, -0.804, 1.368 ], [ 0.048, -0.804, 1.032, 1.64, -0.408, -0.892, 1.652 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476, -0.636, 0.048 ], [ -0.608, 1.912, 1.512, -0.068, 1.552, 0.148, -1.54 ], [ 1.524, 1.108, 0.3, 0.392, -0.384, 1.62, -1.424 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704, -1.492, -1.832 ], [ 1.764, -1.688, 1.556, 1.612, 1.876, -0.032, -0.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572, -1.856, -0.464 ] ]
]
Inputs Statistics: {meanExponent=-0.1649163812740015, negative=31, min=-1.856, max=1.912, mean=0.06050793650793653, count=63, sum=3.8120000000000016, positive=32, stdDev=1.1609433475782607, zeros=0}
Output: [
[ [ -0.24612800000000054, 2.4388959999999997, 0.7840000000000007, -3.27072, -2.6772320000000014 ], [ 2.2020160000000004, -2.6006400000000007, 1.9617599999999988, 11.141919999999995, -4.035104 ], [ 8.038784, -3.000016, -17.415472000000005, 2.73456, -10.585376 ] ],
[ [ 5.467344000000001, 0.5803040000000009, -4.622880000000001, -2.043408000000001, -0.9737439999999998 ], [ -1.0139520000000022, 9.393663999999998, -4.3881760000000005, 0.6679520000000009, 18.145215999999994 ], [ -6.609552000000001, 9.180767999999995, -1.5540000000000003, 14.750991999999998, 13.604224000000002 ] ],
[ [ 5.070736, -12.704479999999998, 2.9870880000000004, -9.798496000000002, -12.032096000000001 ], [ -1.6277600000000003, -13.51464, -2.6166720000000017, 0.7124160000000008, 3.5546720000000014 ], [ 17.698128, -3.0429759999999995, 3.440128000000001, 1.1963519999999996, 0.29596799999999973 ] ]
]
Outputs Statistics: {meanExponent=0.5374743899799378, negative=22, min=-17.415472000000005, max=18.145215999999994, mean=0.3483192888888882, count=45, sum=15.67436799999997, positive=23, stdDev=7.71334115438821, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 2.15 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556, -1.516, 0.996 ], [ 0.496, -0.852, 1.356, -0.768, -1.228, -0.804, 1.368 ], [ 0.048, -0.804, 1.032, 1.64, -0.408, -0.892, 1.652 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476, -0.636, 0.048 ], [ -0.608, 1.912, 1.512, -0.068, 1.552, 0.148, -1.54 ], [ 1.524, 1.108, 0.3, 0.392, -0.384, 1.62, -1.424 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704, -1.492, -1.832 ], [ 1.764, -1.688, 1.556, 1.612, 1.876, -0.032, -0.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572, -1.856, -0.464 ] ]
]
Value Statistics: {meanExponent=-0.1649163812740015, negative=31, min=-1.856, max=1.912, mean=0.06050793650793653, count=63, sum=3.8120000000000016, positive=32, stdDev=1.1609433475782607, zeros=0}
Implemented Feedback: [ [ -1.848, 1.196, 0.0, 0.244, -1.9, 0.0, 0.0, 0.0, ... ], [ -0.688, -1.848, 1.196, -0.816, 0.244, -1.9, 0.0, 0.0, ... ], [ 0.0, -0.688, -1.848, 0.0, -0.816, 0.244, 0.0, 0.0, ... ], [ 1.44, 1.656, 0.0, -1.848, 1.196, 0.0, 0.244, -1.9, ... ], [ -0.632, 1.44, 1.656, -0.688, -1.848, 1.196, -0.816, 0.244, ... ], [ 0.0, -0.632, 1.44, 0.0, -0.688, -1.848, 0.0, -0.816, ... ], [ 0.0, 0.0, 0.0, 1.44, 1.656, 0.0, -1.848, 1.196, ... ], [ 0.0, 0.0, 0.0, -0.632, 1.44, 1.656, -0.688, -1.848, ... ], ... ]
Implemented Statistics: {meanExponent=-0.13470399151896967, negative=795, min=-2.0, max=1.98, mean=0.03559788359788342, count=2835, sum=100.9199999999995, positive=920, stdDev=0.8945164032155165, zeros=1120}
Measured Feedback: [ [ -1.8479999999954089, 1.1959999999966442, 0.0, 0.24400000000035504, -1.8999999999991246, 0.0, 0.0, 0.0, ... ], [ -0.6879999999931385, -1.8479999999954089, 1.196000000005526, -0.8159999999968193, 0.24400000000923683, -1.8999999999991246, 0.0, 0.0, ... ], [ 0.0, -0.6879999999931385, -1.8479999999954089, 0.0, -0.8159999999968193, 0.24400000000035504, 0.0, 0.0, ... ], [ 1.440000000005881, 1.6560000000076514, 0.0, -1.8479999999954089, 1.1959999999966442, 0.0, 0.24399999999147326, -1.8999999999991246, ... ], [ -0.6319999999959691, 1.440000000005881, 1.6560000000076514, -0.6879999999931385, -1.8479999999954089, 1.1959999999966442, -0.8159999999968193, 0.24400000000035504, ... ], [ 0.0, -0.6319999999959691, 1.440000000005881, 0.0, -0.6879999999931385, -1.8480000000042907, 0.0, -0.8159999999968193, ... ], [ 0.0, 0.0, 0.0, 1.440000000005881, 1.6560000000076514, 0.0, -1.8480000000131724, 1.1959999999966442, ... ], [ 0.0, 0.0, 0.0, -0.6319999999959691, 1.440000000005881, 1.6560000000076514, -0.6880000000109021, -1.8479999999954089, ... ], ... ]
Measured Statistics: {meanExponent=-0.13470399151959844, negative=795, min=-2.0000000000131024, max=1.980000000001425, mean=0.0355978835981091, count=2835, sum=100.9200000006393, positive=920, stdDev=0.8945164032153374, zeros=1120}
Feedback Error: [ [ 4.591216296034872E-12, -3.355760114231998E-12, 0.0, 3.5504932327512506E-13, 8.752998326144734E-13, 0.0, 0.0, 0.0, ... ], [ 6.8614003367883925E-12, 4.591216296034872E-12, 5.526024082769254E-12, 3.180677943248611E-12, 9.236833520276377E-12, 8.752998326144734E-13, 0.0, 0.0, ... ], [ 0.0, 6.8614003367883925E-12, 4.591216296034872E-12, 0.0, 3.180677943248611E-12, 3.5504932327512506E-13, 0.0, 0.0, ... ], [ 5.881073406044379E-12, 7.651435041111654E-12, 0.0, 4.591216296034872E-12, -3.355760114231998E-12, 0.0, -8.526734873726127E-12, 8.752998326144734E-13, ... ], [ 4.030886735506556E-12, 5.881073406044379E-12, 7.651435041111654E-12, 6.8614003367883925E-12, 4.591216296034872E-12, -3.355760114231998E-12, 3.180677943248611E-12, 3.5504932327512506E-13, ... ], [ 0.0, 4.030886735506556E-12, 5.881073406044379E-12, 0.0, 6.8614003367883925E-12, -4.29056790096638E-12, 0.0, 3.180677943248611E-12, ... ], [ 0.0, 0.0, 0.0, 5.881073406044379E-12, 7.651435041111654E-12, 0.0, -1.3172352097967632E-11, -3.355760114231998E-12, ... ], [ 0.0, 0.0, 0.0, 4.030886735506556E-12, 5.881073406044379E-12, 7.651435041111654E-12, -1.0902168057214112E-11, 4.591216296034872E-12, ... ], ... ]
Error Statistics: {meanExponent=-11.551418799413938, negative=819, min=-3.323152863998757E-11, max=3.277644822219372E-11, mean=2.2553533115162356E-13, count=2835, sum=6.393926638148528E-10, positive=896, stdDev=5.917531694057124E-12, zeros=1120}

Returns

    {
      "absoluteTol" : {
        "count" : 2835,
        "sum" : 8.843815957337853E-9,
        "min" : 0.0,
        "max" : 3.323152863998757E-11,
        "sumOfSquare" : 9.941791476390299E-20,
        "standardDeviation" : 5.0335567599742666E-12,
        "average" : 3.119511801530107E-12
      },
      "relativeTol" : {
        "count" : 1715,
        "sum" : 1.3965971525136381E-8,
        "min" : 7.308420798198091E-15,
        "max" : 8.182894472887003E-10,
        "sumOfSquare" : 2.2485183176545716E-18,
        "standardDeviation" : 3.528135555268035E-11,
        "average" : 8.143423629817132E-12
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 2.94 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Learning Gradient for weight setByCoord 0
Weights: [ -0.632, 1.44, 1.656, -0.688, -1.848, 1.196, -0.816, 0.244, ... ]
Implemented Gradient: [ [ -0.608, 1.764, 0.0, 1.524, 1.208, 0.0, 0.0, 0.0, ... ], [ 0.496, -0.608, 1.764, 0.048, 1.524, 1.208, 0.0, 0.0, ... ], [ 0.0, 0.496, -0.608, 0.0, 0.048, 1.524, 0.0, 0.0, ... ], [ 0.7, -0.128, 0.0, -0.608, 1.764, 0.0, 1.524, 1.208, ... ], [ 0.08, 0.7, -0.128, 0.496, -0.608, 1.764, 0.048, 1.524, ... ], [ 0.0, 0.08, 0.7, 0.0, 0.496, -0.608, 0.0, 0.048, ... ], [ 0.0, 0.0, 0.0, 0.7, -0.128, 0.0, -0.608, 1.764, ... ], [ 0.0, 0.0, 0.0, 0.08, 0.7, -0.128, 0.496, -0.608, ... ], ... ]
Implemented Statistics: {meanExponent=-0.15671952155452043, negative=815, min=-1.856, max=1.912, mean=0.011097001763668473, count=19845, sum=220.22000000000085, positive=900, stdDev=0.345415427419215, zeros=18130}
Measured Gradient: [ [ -0.6079999999997199, 1.7640000000085365, 0.0, 1.5240000000016352, 1.2079999999947688, 0.0, 0.0, 0.0, ... ], [ 0.496000000005381, -0.6079999999997199, 1.7639999999996547, 0.04800000000138027, 1.524000000010517, 1.2080000000036506, 0.0, 0.0, ... ], [ 0.0, 0.496000000005381, -0.6079999999997199, 0.0, 0.048000000010262056, 1.5240000000016352, 0.0, 0.0, ... ], [ 0.7000000000090267, -0.12800000000368073, 0.0, -0.6079999999997199, 1.7640000000085365, 0.0, 1.5239999999927534, 1.2080000000036506, ... ], [ 0.08000000001118224, 0.7000000000090267, -0.12799999999479894, 0.496000000005381, -0.6079999999997199, 1.7639999999996547, 0.04799999999249849, 1.5240000000016352, ... ], [ 0.0, 0.07999999999341867, 0.700000000000145, 0.0, 0.496000000005381, -0.6079999999997199, 0.0, 0.04800000000138027, ... ], [ 0.0, 0.0, 0.0, 0.700000000000145, -0.12800000000368073, 0.0, -0.6079999999997199, 1.7639999999996547, ... ], [ 0.0, 0.0, 0.0, 0.08000000000230045, 0.7000000000090267, -0.12800000000368073, 0.49599999998761746, -0.6079999999997199, ... ], ... ]
Measured Statistics: {meanExponent=-0.1567195215547656, negative=815, min=-1.8560000000178434, max=1.9120000000327764, mean=0.011097001763705802, count=19845, sum=220.22000000074163, positive=900, stdDev=0.34541542741919523, zeros=18130}
Gradient Error: [ [ 2.80109269112927E-13, 8.536504836342829E-12, 0.0, 1.6351364706679306E-12, -5.2311488474288126E-12, 0.0, 0.0, 0.0, ... ], [ 5.381028955753209E-12, 2.80109269112927E-13, -3.452793606584237E-13, 1.380270897577418E-12, 1.0516920667669183E-11, 3.65063534957244E-12, 0.0, 0.0, ... ], [ 0.0, 5.381028955753209E-12, 2.80109269112927E-13, 0.0, 1.026205509457867E-11, 1.6351364706679306E-12, 0.0, 0.0, ... ], [ 9.026779324017298E-12, -3.6807223935397815E-12, 0.0, 2.80109269112927E-13, 8.536504836342829E-12, 0.0, -7.246647726333322E-12, 3.65063534957244E-12, ... ], [ 1.1182235692963616E-11, 9.026779324017298E-12, 5.201061803461471E-12, 5.381028955753209E-12, 2.80109269112927E-13, -3.452793606584237E-13, -7.501513299423834E-12, 1.6351364706679306E-12, ... ], [ 0.0, -6.581332701038889E-12, 1.4499512701604544E-13, 0.0, 5.381028955753209E-12, 2.80109269112927E-13, 0.0, 1.380270897577418E-12, ... ], [ 0.0, 0.0, 0.0, 1.4499512701604544E-13, -3.6807223935397815E-12, 0.0, 2.80109269112927E-13, -3.452793606584237E-13, ... ], [ 0.0, 0.0, 0.0, 2.3004514959623634E-12, 9.026779324017298E-12, -3.6807223935397815E-12, -1.2382539438249296E-11, 2.80109269112927E-13, ... ], ... ]
Error Statistics: {meanExponent=-11.619039844070686, negative=872, min=-3.1531222077774146E-11, max=3.277644822219372E-11, mean=3.7363840500616414E-14, count=19845, sum=7.414854147347327E-10, positive=843, stdDev=2.22080846871981E-12, zeros=18130}

Returns

    {
      "absoluteTol" : {
        "count" : 22680,
        "sum" : 1.7420134203194326E-8,
        "min" : 0.0,
        "max" : 3.323152863998757E-11,
        "sumOfSquare" : 1.9732096611194092E-19,
        "standardDeviation" : 2.847852993526026E-12,
        "average" : 7.680835186593618E-13
      },
      "relativeTol" : {
        "count" : 3430,
        "sum" : 2.9356303582564127E-8,
        "min" : 7.308420798198091E-15,
        "max" : 8.182894472887003E-10,
        "sumOfSquare" : 3.515412769967346E-18,
        "standardDeviation" : 3.084883341793745E-11,
        "average" : 8.558689091126567E-12
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 7.6808e-13 +- 2.8479e-12 [0.0000e+00 - 3.3232e-11] (22680#)
relativeTol: 8.5587e-12 +- 3.0849e-11 [7.3084e-15 - 8.1829e-10] (3430#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.03 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=7.6808e-13 +- 2.8479e-12 [0.0000e+00 - 3.3232e-11] (22680#), relativeTol=8.5587e-12 +- 3.0849e-11 [7.3084e-15 - 8.1829e-10] (3430#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "6.776",
      "gc_time": "1.457"
    },
    "created_on": 1586747380864,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "IrregularTest",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.IrregularTest",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/IrregularTest/derivativeTest/202004130940",
    "id": "6e49c90e-a914-4359-9726-165e05a4f8fd",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ConvolutionLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
      "javaDoc": ""
    }
  }