1. Test Modules
  2. Network Diagram
  3. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  4. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Network Diagram

This is a network apply the following layout:

LayerTests.java:203 executed in 0.07 seconds (0.000 gc):

    return Graphviz.fromGraph((Graph) TestUtil.toGraph(((DAGNetwork) layer).addRef())).height(400).width(600)
        .render(Format.PNG).toImage();
Logging
executing command [/bin/sh, -c, dot -Tsvg /tmp/GraphvizJava/DotEngine6511394909320047458/dotfile.dot -ooutfile.svg]

Returns

Result

Using Seed 3970727995878793216

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging
Inputs: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556, -1.516, 0.996 ], [ 0.496, -0.852, 1.356, -0.768, -1.228, -0.804, 1.368 ], [ 0.048, -0.804, 1.032, 1.64, -0.408, -0.892, 1.652 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476, -0.636, 0.048 ], [ -0.608, 1.912, 1.512, -0.068, 1.552, 0.148, -1.54 ], [ 1.524, 1.108, 0.3, 0.392, -0.384, 1.62, -1.424 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704, -1.492, -1.832 ], [ 1.764, -1.688, 1.556, 1.612, 1.876, -0.032, -0.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572, -1.856, -0.464 ] ]
]
Inputs Statistics: {meanExponent=-0.1649163812740015, negative=31, min=-1.856, max=1.912, mean=0.06050793650793653, count=63, sum=3.8120000000000016, positive=32, stdDev=1.1609433475782607, zeros=0}
Output: [
[ [ -4.095007419586182, -3.187039375305176, -0.5018725395202637, -6.695104122161865, 1.4703993797302246 ], [ -0.28078460693359375, 3.973599433898926, -5.366574764251709, -8.206352233886719, -0.8971853256225586 ], [ 5.114239692687988, -4.592223167419434, -5.896639823913574, -2.7007668018341064, 8.015151977539062 ] ],
[ [ 10.951376914978027, -8.50948715209961, 3.8426103591918945, 3.931550979614258, -19.3441104888916 ], [ -6.528129577636719, -2.1880035400390625, 4.742222785949707, -9.325889587402344, -8.849710464477539 ], [ 12.142208099365234, 6.341232776641846, 9.376224517822266, 5.648559093475342, -9.426706314086914 ] ],
[ [ -1.5146896839141846, 4.236288070678711, 14.20607852935791, -17.819229125976562, -18.73579216003418 ], [ -11.513566970825195, 18.00289535522461, 5.891967296600342, 2.4112472534179688, -0.16446399688720703 ], [ -2.4938879013061523, 6.3666253089904785, -3.8166399002075195, 12.433979034423828, 0.8532464504241943 ] ]
]
Outputs Statistics: {meanExponent=0.6594331188580219, negative=25, min=-19.3441104888916, max=18.00289535522461, mean=-0.5044034163157145, count=45, sum=-22.698153734207153, positive=20, stdDev=8.382306440062722, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 2.34 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Feedback for input 0
Inputs Values: [
[ [ 0.08, -1.72, -0.712, -0.176, -0.556, -1.516, 0.996 ], [ 0.496, -0.852, 1.356, -0.768, -1.228, -0.804, 1.368 ], [ 0.048, -0.804, 1.032, 1.64, -0.408, -0.892, 1.652 ] ],
[ [ 0.7, -1.028, 1.048, 1.556, -1.476, -0.636, 0.048 ], [ -0.608, 1.912, 1.512, -0.068, 1.552, 0.148, -1.54 ], [ 1.524, 1.108, 0.3, 0.392, -0.384, 1.62, -1.424 ] ],
[ [ -0.128, -0.384, -1.616, 0.788, 1.704, -1.492, -1.832 ], [ 1.764, -1.688, 1.556, 1.612, 1.876, -0.032, -0.876 ], [ 1.208, 0.028, 0.636, 0.092, -1.572, -1.856, -0.464 ] ]
]
Value Statistics: {meanExponent=-0.1649163812740015, negative=31, min=-1.856, max=1.912, mean=0.06050793650793653, count=63, sum=3.8120000000000016, positive=32, stdDev=1.1609433475782607, zeros=0}
Implemented Feedback: [ [ -1.4119999408721924, -0.019999999552965164, 0.0, 0.2639999985694885, -0.4519999921321869, 0.0, 0.0, 0.0, ... ], [ 0.5199999809265137, -1.4119999408721924, -0.019999999552965164, 1.0520000457763672, 0.2639999985694885, -0.4519999921321869, 0.0, 0.0, ... ], [ 0.0, 0.5199999809265137, -1.4119999408721924, 0.0, 1.0520000457763672, 0.2639999985694885, 0.0, 0.0, ... ], [ 0.9959999918937683, -1.9199999570846558, 0.0, -1.4119999408721924, -0.019999999552965164, 0.0, 0.2639999985694885, -0.4519999921321869, ... ], [ -0.5479999780654907, 0.9959999918937683, -1.9199999570846558, 0.5199999809265137, -1.4119999408721924, -0.019999999552965164, 1.0520000457763672, 0.2639999985694885, ... ], [ 0.0, -0.5479999780654907, 0.9959999918937683, 0.0, 0.5199999809265137, -1.4119999408721924, 0.0, 1.0520000457763672, ... ], [ 0.0, 0.0, 0.0, 0.9959999918937683, -1.9199999570846558, 0.0, -1.4119999408721924, -0.019999999552965164, ... ], [ 0.0, 0.0, 0.0, -0.5479999780654907, 0.9959999918937683, -1.9199999570846558, 0.5199999809265137, -1.4119999408721924, ... ], ... ]
Implemented Statistics: {meanExponent=-0.13081934851135346, negative=824, min=-2.0, max=1.9800000190734863, mean=0.01573474344647961, count=2835, sum=44.60799767076969, positive=891, stdDev=0.8962130373554171, zeros=1120}
Measured Feedback: [ [ -1.4162063598632812, -0.019073486328125, 0.0, 0.26226043701171875, -0.4482269287109375, 0.0, 0.0, 0.0, ... ], [ 0.5245208740234375, -1.4209747314453125, -0.011920928955078125, 1.049041748046875, 0.26226043701171875, -0.457763671875, 0.0, 0.0, ... ], [ 0.0, 0.5340576171875, -1.4162063598632812, 0.0, 1.0538101196289062, 0.26702880859375, 0.0, 0.0, ... ], [ 0.9965896606445312, -1.926422119140625, 0.0, -1.41143798828125, -0.0095367431640625, 0.0, 0.26226043701171875, -0.438690185546875, ... ], [ -0.5483627319335938, 0.9918212890625, -1.9121170043945312, 0.5197525024414062, -1.41143798828125, -0.019073486328125, 1.0538101196289062, 0.26702880859375, ... ], [ 0.0, -0.5435943603515625, 0.9918212890625, 0.0, 0.514984130859375, -1.41143798828125, 0.0, 1.049041748046875, ... ], [ 0.0, 0.0, 0.0, 0.9965896606445312, -1.9121170043945312, 0.0, -1.41143798828125, -0.0095367431640625, ... ], [ 0.0, 0.0, 0.0, -0.5435943603515625, 0.9965896606445312, -1.926422119140625, 0.5197525024414062, -1.4019012451171875, ... ], ... ]
Measured Statistics: {meanExponent=-0.12449242071872545, negative=823, min=-2.002716064453125, max=1.9931793212890625, mean=0.015570793622805749, count=2835, sum=44.1431999206543, positive=886, stdDev=0.8965959007172701, zeros=1126}
Feedback Error: [ [ -0.004206418991088867, 9.265132248401642E-4, 0.0, -0.0017395615577697754, 0.0037730634212493896, 0.0, 0.0, 0.0, ... ], [ 0.004520893096923828, -0.008974790573120117, 0.00807907059788704, -0.0029582977294921875, -0.0017395615577697754, -0.00576367974281311, 0.0, 0.0, ... ], [ 0.0, 0.014057636260986328, -0.004206418991088867, 0.0, 0.0018100738525390625, 0.0030288100242614746, 0.0, 0.0, ... ], [ 5.896687507629395E-4, -0.006422162055969238, 0.0, 5.619525909423828E-4, 0.010463256388902664, 0.0, -0.0017395615577697754, 0.01330980658531189, ... ], [ -3.6275386810302734E-4, -0.0041787028312683105, 0.007882952690124512, -2.474784851074219E-4, 5.619525909423828E-4, 9.265132248401642E-4, 0.0018100738525390625, 0.0030288100242614746, ... ], [ 0.0, 0.004405617713928223, -0.0041787028312683105, 0.0, -0.005015850067138672, 5.619525909423828E-4, 0.0, -0.0029582977294921875, ... ], [ 0.0, 0.0, 0.0, 5.896687507629395E-4, 0.007882952690124512, 0.0, 5.619525909423828E-4, 0.010463256388902664, ... ], [ 0.0, 0.0, 0.0, 0.004405617713928223, 5.896687507629395E-4, -0.006422162055969238, -2.474784851074219E-4, 0.010098695755004883, ... ], ... ]
Error Statistics: {meanExponent=-2.4438463970483504, negative=840, min=-0.04438328742980957, max=0.037038207054138184, mean=-1.6394982367386052E-4, count=2835, sum=-0.4647977501153946, positive=875, stdDev=0.006880768591366095, zeros=1120}

Returns

    {
      "absoluteTol" : {
        "count" : 2835,
        "sum" : 10.856813665479422,
        "min" : 0.0,
        "max" : 0.04438328742980957,
        "sumOfSquare" : 0.13429921162565744,
        "standardDeviation" : 0.005718941883265693,
        "average" : 0.0038295639031673445
      },
      "relativeTol" : {
        "count" : 1715,
        "sum" : 21.624646797209753,
        "min" : 1.0593231560626653E-6,
        "max" : 1.0,
        "sumOfSquare" : 7.664713489528077,
        "standardDeviation" : 0.0656523472409168,
        "average" : 0.012609123496915307
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 3.76 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging
Learning Gradient for weight setByCoord 0
Weights: [ -0.548, 0.996, -1.92, 0.52, -1.412, -0.02, 1.052, 0.264, ... ]
Implemented Gradient: [ [ -0.6079999804496765, 1.7640000581741333, 0.0, 1.5240000486373901, 1.2079999446868896, 0.0, 0.0, 0.0, ... ], [ 0.4959999918937683, -0.6079999804496765, 1.7640000581741333, 0.04800000041723251, 1.5240000486373901, 1.2079999446868896, 0.0, 0.0, ... ], [ 0.0, 0.4959999918937683, -0.6079999804496765, 0.0, 0.04800000041723251, 1.5240000486373901, 0.0, 0.0, ... ], [ 0.699999988079071, -0.12800000607967377, 0.0, -0.6079999804496765, 1.7640000581741333, 0.0, 1.5240000486373901, 1.2079999446868896, ... ], [ 0.07999999821186066, 0.699999988079071, -0.12800000607967377, 0.4959999918937683, -0.6079999804496765, 1.7640000581741333, 0.04800000041723251, 1.5240000486373901, ... ], [ 0.0, 0.07999999821186066, 0.699999988079071, 0.0, 0.4959999918937683, -0.6079999804496765, 0.0, 0.04800000041723251, ... ], [ 0.0, 0.0, 0.0, 0.699999988079071, -0.12800000607967377, 0.0, -0.6079999804496765, 1.7640000581741333, ... ], [ 0.0, 0.0, 0.0, 0.07999999821186066, 0.699999988079071, -0.12800000607967377, 0.4959999918937683, -0.6079999804496765, ... ], ... ]
Implemented Statistics: {meanExponent=-0.15671952127919558, negative=815, min=-1.8559999465942383, max=1.9119999408721924, mean=0.011097001637836402, count=19845, sum=220.2199975028634, positive=900, stdDev=0.3454154267738122, zeros=18130}
Measured Gradient: [ [ -0.6008148193359375, 1.7642974853515625, 0.002384185791015625, 1.52587890625, 1.220703125, 0.0, -0.00476837158203125, 0.0, ... ], [ 0.49591064453125, -0.6103515625, 1.773834228515625, 0.0476837158203125, 1.5211105346679688, 1.201629638671875, 0.0, 0.0, ... ], [ 0.00476837158203125, 0.49591064453125, -0.6008148193359375, 0.0, 0.05245208740234375, 1.52587890625, 0.0, 0.019073486328125, ... ], [ 0.7009506225585938, -0.1239776611328125, -0.007152557373046875, -0.6055831909179688, 1.773834228515625, 0.0, 1.5306472778320312, 1.201629638671875, ... ], [ 0.08106231689453125, 0.6961822509765625, -0.1239776611328125, 0.49591064453125, -0.6055831909179688, 1.773834228515625, 0.05245208740234375, 1.52587890625, ... ], [ 0.0, 0.0858306884765625, 0.6961822509765625, 0.0, 0.5102157592773438, -0.6103515625, 0.00476837158203125, 0.03814697265625, ... ], [ 0.00476837158203125, 0.0, -0.002384185791015625, 0.7009506225585938, -0.11920928955078125, -0.0095367431640625, -0.6055831909179688, 1.7642974853515625, ... ], [ -0.00476837158203125, 0.0, 0.0, 0.0858306884765625, 0.7009506225585938, -0.133514404296875, 0.5006790161132812, -0.6103515625, ... ], ... ]
Measured Statistics: {meanExponent=-0.6671964686688886, negative=1104, min=-1.86920166015625, max=1.926422119140625, mean=0.011090999227472384, count=19845, sum=220.10087966918945, positive=1228, stdDev=0.3455561506647067, zeros=17513}
Gradient Error: [ [ 0.007185161113739014, 2.974271774291992E-4, 0.002384185791015625, 0.0018788576126098633, 0.012703180313110352, 0.0, -0.00476837158203125, 0.0, ... ], [ -8.934736251831055E-5, -0.0023515820503234863, 0.0098341703414917, -3.1628459692001343E-4, -0.0028895139694213867, -0.0063703060150146484, 0.0, 0.0, ... ], [ 0.00476837158203125, -8.934736251831055E-5, 0.007185161113739014, 0.0, 0.004452086985111237, 0.0018788576126098633, 0.0, 0.019073486328125, ... ], [ 9.506344795227051E-4, 0.004022344946861267, -0.007152557373046875, 0.0024167895317077637, 0.0098341703414917, 0.0, 0.006647229194641113, -0.0063703060150146484, ... ], [ 0.0010623186826705933, -0.003817737102508545, 0.004022344946861267, -8.934736251831055E-5, 0.0024167895317077637, 0.0098341703414917, 0.004452086985111237, 0.0018788576126098633, ... ], [ 0.0, 0.005830690264701843, -0.003817737102508545, 0.0, 0.01421576738357544, -0.0023515820503234863, 0.00476837158203125, -0.009853027760982513, ... ], [ 0.00476837158203125, 0.0, -0.002384185791015625, 9.506344795227051E-4, 0.008790716528892517, -0.0095367431640625, 0.0024167895317077637, 2.974271774291992E-4, ... ], [ -0.00476837158203125, 0.0, 0.0, 0.005830690264701843, 9.506344795227051E-4, -0.005514398217201233, 0.0046790242195129395, -0.0023515820503234863, ... ], ... ]
Error Statistics: {meanExponent=-2.3077076217652026, negative=1164, min=-0.04279828071594238, max=0.03814697265625, mean=-6.002410364018847E-6, count=19845, sum=-0.11911783367395401, positive=1168, stdDev=0.003361420512373179, zeros=17513}

Returns

    {
      "absoluteTol" : {
        "count" : 22680,
        "sum" : 28.658945973962545,
        "min" : 0.0,
        "max" : 0.04438328742980957,
        "sumOfSquare" : 0.3585315159213849,
        "standardDeviation" : 0.003769817959631333,
        "average" : 0.0012636219565239218
      },
      "relativeTol" : {
        "count" : 4047,
        "sum" : 658.4989922489947,
        "min" : 1.0593231560626653E-6,
        "max" : 1.0,
        "sumOfSquare" : 626.3763004749129,
        "standardDeviation" : 0.3581898725091437,
        "average" : 0.16271287181838268
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging
Finite-Difference Derivative Accuracy:
absoluteTol: 1.2636e-03 +- 3.7698e-03 [0.0000e+00 - 4.4383e-02] (22680#)
relativeTol: 1.6271e-01 +- 3.5819e-01 [1.0593e-06 - 1.0000e+00] (4047#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.03 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=1.2636e-03 +- 3.7698e-03 [0.0000e+00 - 4.4383e-02] (22680#), relativeTol=1.6271e-01 +- 3.5819e-01 [1.0593e-06 - 1.0000e+00] (4047#)}OK
  {
    "result": "OK",
    "performance": {
      "execution_time": "7.814",
      "gc_time": "1.510"
    },
    "created_on": 1586747736467,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "IrregularTest_Float",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayerTest.IrregularTest_Float",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer/IrregularTest_Float/derivativeTest/202004131536",
    "id": "39dc24fb-453d-4c1a-80f0-f035fc9227bf",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "ConvolutionLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.conv.ConvolutionLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/conv/ConvolutionLayer.java",
      "javaDoc": ""
    }
  }