1. Test Modules
  2. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  3. Results

Subreport: Logs for com.simiacryptus.ref.lang.ReferenceCountingBase

Test Modules

Using Seed 5389562388683508736

Differential Validation

SingleDerivativeTester.java:101 executed in 0.00 seconds (0.000 gc):

        log.info(RefString.format("Inputs: %s", prettyPrint(inputPrototype)));
        log.info(RefString.format("Inputs Statistics: %s", printStats(inputPrototype)));
        log.info(RefString.format("Output: %s", outputPrototype.prettyPrint()));
        assert outputPrototype != null;
        log.info(RefString.format("Outputs Statistics: %s", outputPrototype.getScalarStatistics()));
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype)));
Logging 
Inputs: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Inputs Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Output: [
[ [ 0.048788736914720444, 0.02452025437730663, 0.15073140608686714 ], [ 0.039626555434423484, 0.04725221283323708, 0.016110955159665548 ] ],
[ [ 0.09069475117703761, 0.26282791049567256, 0.008064723969738768 ], [ 0.07395815684906859, 0.20674775706180246, 0.030676579640459757 ] ]
]
Outputs Statistics: {meanExponent=-1.2757787479650196, negative=0, min=0.008064723969738768, max=0.26282791049567256, mean=0.08333333333333336, count=12, sum=1.0000000000000002, positive=12, stdDev=0.07797189147045935, zeros=0}

Feedback Validation

We validate the agreement between the implemented derivative of the inputs apply finite difference estimations:

SingleDerivativeTester.java:117 executed in 0.17 seconds (0.000 gc):

        return testFeedback(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));
Logging 
Feedback for input 0
Inputs Values: [
[ [ 0.08, -0.608, 1.208 ], [ -0.128, 0.048, -1.028 ] ],
[ [ 0.7, 1.764, -1.72 ], [ 0.496, 1.524, -0.384 ] ]
]
Value Statistics: {meanExponent=-0.3033810201823816, negative=5, min=-1.72, max=1.764, mean=0.16266666666666665, count=12, sum=1.952, positive=7, stdDev=0.9945442283891763, zeros=0}
Implemented Feedback: [ [ 0.04640839606498664, -0.00442488235472252, -0.0019333295879266732, -0.0036083250572068373, -0.0011963122398964356, -0.012823041779019061, -0.002305375780559181, -0.010086961926996816, ... ], [ -0.00442488235472252, 0.08246921328597284, -0.0035939205851281252, -0.006707616632938596, -0.002223858369547492, -0.023837111944785733, -0.00428552768547486, -0.018750936383130794, ... ], [ -0.0019333295879266732, -0.0035939205851281252, 0.038056291538826045, -0.0029307070022074034, -9.716532193485062E-4, -0.010414964764970462, -0.0018724424312354458, -0.008192701456152234, ... ], [ -0.0036083250572068373, -0.006707616632938596, -0.0029307070022074034, 0.06848834788455715, -0.0018134728192159043, -0.019438267828751913, -0.0034946865681861196, -0.015290683044969914, ... ], [ -0.0011963122398964356, -0.002223858369547492, -9.716532193485062E-4, -0.0018134728192159043, 0.023919011502578802, -0.00644460722280987, -0.0011586362785616059, -0.005069507595092989, ... ], [ -0.012823041779019061, -0.023837111944785733, -0.010414964764970462, -0.019438267828751913, -0.00644460722280987, 0.19374939996015128, -0.012419200365256506, -0.05433908098822047, ... ], [ -0.002305375780559181, -0.00428552768547486, -0.0018724424312354458, -0.0034946865681861196, -0.0011586362785616059, -0.012419200365256506, 0.045019441215599545, -0.009769289019478683, ... ], [ -0.010086961926996816, -0.018750936383130794, -0.008192701456152234, -0.015290683044969914, -0.005069507595092989, -0.05433908098822047, -0.009769289019478683, 0.16400312201171635, ... ], ... ]
Implemented Statistics: {meanExponent=-2.4485330373248084, negative=132, min=-0.05433908098822047, max=0.19374939996015128, mean=-4.577742506049908E-19, count=144, sum=-6.591949208711867E-17, positive=12, stdDev=0.028493904476059595, zeros=0}
Measured Feedback: [ [ 0.04641049011988585, -0.004425082015813153, -0.001933416824209666, -0.003608487873230093, -0.0011963662202921221, -0.012823620384461165, -0.0023054798044958025, -0.010087417074144955, ... ], [ -0.004425063471202861, 0.08247258886359687, -0.003594067689224878, -0.006707891184870585, -0.0022239493951276, -0.023838087630512206, -0.004285703098030891, -0.0187517038846563, ... ], [ -0.0019334185958480576, -0.003594086044472755, 0.03805804359824361, -0.002930841928039385, -9.71697952968642E-4, -0.0104154442559512, -0.001872528635968318, -0.008193078637197537, ... ], [ -0.0036084787904955284, -0.006707902412139699, -0.0029308318654636123, 0.06849126584204668, -0.0018135500825730566, -0.019439095999551625, -0.0034948354599162457, -0.015291334507228083, ... ], [ -0.0011963691239069707, -0.002223964112868515, -9.716994209263419E-4, -0.0018135590489076758, 0.023920148837126842, -0.00644491366053046, -0.00115869137111424, -0.005069748647557049, ... ], [ -0.012823345902218675, -0.023837677288057746, -0.010415211776082511, -0.019438728844772823, -0.00644476006922412, 0.19375399510301872, -0.012419494910681506, -0.05434036974477641, ... ], [ -0.002305480158795725, -0.004285721717095514, -0.0018725272079439526, -0.0034948447936999916, -0.0011586887370060306, -0.012419762657200906, 0.04502147951550295, -0.009769731333930665, ... ], [ -0.010087257729662902, -0.01875148625901657, -0.008192941709159185, -0.01529113144799199, -0.005069656259654165, -0.054340674495167995, -0.009769575506288208, 0.1640079314446652, ... ], ... ]
Measured Statistics: {meanExponent=-2.4485149416836522, negative=132, min=-0.054340674495167995, max=0.19375399510301872, mean=-2.9393925565162566E-14, count=144, sum=-4.232725281383409E-12, positive=12, stdDev=0.02849478247539498, zeros=0}
Feedback Error: [ [ 2.094054899211273E-6, -1.9966109063296322E-7, -8.723628299291478E-8, -1.628160232556436E-7, -5.398039568658454E-8, -5.786054421037801E-7, -1.0402393662168469E-7, -4.5514714813905954E-7, ... ], [ -1.8111648034113692E-7, 3.3755776240318447E-6, -1.4710409675273983E-7, -2.7455193198858835E-7, -9.102558010815054E-8, -9.75685726472414E-7, -1.7541255603083228E-7, -7.675015255073636E-7, ... ], [ -8.900792138446037E-8, -1.654593446299285E-7, 1.7520594175618465E-6, -1.3492583198155064E-7, -4.473362013581936E-8, -4.794909807381148E-7, -8.620473287220161E-8, -3.7718104530307106E-7, ... ], [ -1.537332886911852E-7, -2.857792011029883E-7, -1.248632562089215E-7, 2.917957489523282E-6, -7.726335715229488E-8, -8.281707997125076E-7, -1.4889173012610016E-7, -6.514622581683255E-7, ... ], [ -5.688401053514404E-8, -1.0574332102303297E-7, -4.620157783566045E-8, -8.622969177151524E-8, 1.1373345480403252E-6, -3.064377205900734E-7, -5.509255263406662E-8, -2.4105246405978914E-7, ... ], [ -3.041231996132876E-7, -5.653432720124918E-7, -2.470111120488755E-7, -4.610160209105041E-7, -1.5284641425011153E-7, 4.5951428674362305E-6, -2.945454249999452E-7, -1.2887565559380199E-6, ... ], [ -1.0437823654441819E-7, -1.940316206542475E-7, -8.47767085067775E-8, -1.5822551387200193E-7, -5.245844442473488E-8, -5.622919444000368E-7, 2.03829990340354E-6, -4.42314451981618E-7, ... ], [ -2.958026660854901E-7, -5.498758857760966E-7, -2.402530069507691E-7, -4.4840302207589666E-7, -1.4866456117621635E-7, -1.593506947526746E-6, -2.8648680952536587E-7, 4.809432948837955E-6, ... ], ... ]
Error Statistics: {meanExponent=-6.838059808788202, negative=132, min=-1.593506947526746E-6, max=4.809432948837955E-6, mean=-2.939321518686413E-14, count=144, sum=-4.232622986908435E-12, positive=12, stdDev=9.020075961519132E-7, zeros=0}

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 5.9494297437677186E-5,
        "min" : 6.287392736846085E-9,
        "max" : 4.809432948837955E-6,
        "sumOfSquare" : 1.1716094930626854E-10,
        "standardDeviation" : 8.0182340883719E-7,
        "average" : 4.131548433172027E-7
      },
      "relativeTol" : {
        "count" : 144,
        "sum" : 0.0030000062574920858,
        "min" : 1.1858319944779731E-5,
        "max" : 2.4597098460141223E-5,
        "sumOfSquare" : 6.468909438263282E-8,
        "standardDeviation" : 3.898747709845776E-6,
        "average" : 2.0833376788139486E-5
      }
    }

Learning Validation

We validate the agreement between the implemented derivative of the internal weights apply finite difference estimations:

SingleDerivativeTester.java:133 executed in 0.00 seconds (0.000 gc):

        return testLearning(
            statistics,
            component.addRef(),
            RefUtil.addRef(inputPrototype),
            outputPrototype.addRef());
      },
      outputPrototype.addRef(),
      RefUtil.addRef(inputPrototype),
      component.addRef()));

Returns

    {
      "absoluteTol" : {
        "count" : 144,
        "sum" : 5.9494297437677186E-5,
        "min" : 6.287392736846085E-9,
        "max" : 4.809432948837955E-6,
        "sumOfSquare" : 1.1716094930626854E-10,
        "standardDeviation" : 8.0182340883719E-7,
        "average" : 4.131548433172027E-7
      },
      "relativeTol" : {
        "count" : 144,
        "sum" : 0.0030000062574920858,
        "min" : 1.1858319944779731E-5,
        "max" : 2.4597098460141223E-5,
        "sumOfSquare" : 6.468909438263282E-8,
        "standardDeviation" : 3.898747709845776E-6,
        "average" : 2.0833376788139486E-5
      }
    }

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

SingleDerivativeTester.java:148 executed in 0.00 seconds (0.000 gc):

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toStream(inputPrototype), outputPrototype));
    log.info(RefString.format("Finite-Difference Derivative Accuracy:"));
    log.info(RefString.format("absoluteTol: %s", statistics.absoluteTol));
    log.info(RefString.format("relativeTol: %s", statistics.relativeTol));
Logging 
Finite-Difference Derivative Accuracy:
absoluteTol: 4.1315e-07 +- 8.0182e-07 [6.2874e-09 - 4.8094e-06] (144#)
relativeTol: 2.0833e-05 +- 3.8987e-06 [1.1858e-05 - 2.4597e-05] (144#)

Frozen and Alive Status

SingleDerivativeTester.java:156 executed in 0.01 seconds (0.000 gc):

    testFrozen(component.addRef(), RefUtil.addRef(inputPrototype));
    testUnFrozen(component.addRef(), RefUtil.addRef(inputPrototype));

LayerTests.java:425 executed in 0.00 seconds (0.000 gc):

    throwException(exceptions.addRef());

Results

classdetailsresult
com.simiacryptus.mindseye.test.unit.SingleDerivativeTesterToleranceStatistics{absoluteTol=4.1315e-07 +- 8.0182e-07 [6.2874e-09 - 4.8094e-06] (144#), relativeTol=2.0833e-05 +- 3.8987e-06 [1.1858e-05 - 2.4597e-05] (144#)}OK
 
  {
    "result": "OK",
    "performance": {
      "execution_time": "0.358",
      "gc_time": "0.136"
    },
    "created_on": 1586742375475,
    "file_name": "derivativeTest",
    "report": {
      "simpleName": "Basic",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.SoftmaxLayerTest.Basic",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/test/java/com/simiacryptus/mindseye/layers/cudnn/SoftmaxLayerTest.java",
      "javaDoc": ""
    },
    "archive": "s3://code.simiacrypt.us/tests/com/simiacryptus/mindseye/layers/cudnn/SoftmaxActivationLayer/Basic/derivativeTest/202004134615",
    "id": "67493c30-9aad-4d56-8300-441eaeafffda",
    "report_type": "Components",
    "display_name": "Derivative Validation",
    "target": {
      "simpleName": "SoftmaxActivationLayer",
      "canonicalName": "com.simiacryptus.mindseye.layers.cudnn.SoftmaxActivationLayer",
      "link": "https://github.com/SimiaCryptus/mindseye-cudnn/tree/59d5b3318556370acb2d83ee6ec123ce0fc6974f/src/main/java/com/simiacryptus/mindseye/layers/cudnn/SoftmaxActivationLayer.java",
      "javaDoc": ""
    }
  }