====== Perplexity statistics ======
Mean PPL(Q)                   :   6.983531 ±   0.043462
Mean PPL(base)                :   6.554978 ±   0.040159
Cor(ln(PPL(Q)), ln(PPL(base))):  98.37%
Mean ln(PPL(Q)/PPL(base))     :   0.063330 ±   0.001119
Mean PPL(Q)/PPL(base)         :   1.065378 ±   0.001192
Mean PPL(Q)-PPL(base)         :   0.428552 ±   0.008232

====== KL divergence statistics ======
Mean    KLD:   0.071002 ±   0.000463
Maximum KLD:   7.889337
99.9%   KLD:   2.299609
99.0%   KLD:   0.707767
99.0%   KLD:   0.707767
Median  KLD:   0.033880
10.0%   KLD:   0.001714
 5.0%   KLD:   0.000489
 1.0%   KLD:   0.000073
Minimum KLD:  -0.000059

====== Token probability statistics ======
Mean    Δp: -1.167 ± 0.021 %
Maximum Δp: 85.060%
99.9%   Δp: 33.978%
99.0%   Δp: 16.661%
95.0%   Δp:  7.783%
90.0%   Δp:  4.533%
75.0%   Δp:  0.828%
Median  Δp: -0.060%
25.0%   Δp: -2.077%
10.0%   Δp: -7.263%
 5.0%   Δp: -12.280%
 1.0%   Δp: -32.484%
 0.1%   Δp: -77.734%
Minimum Δp: -98.234%
RMS Δp    :  8.106 ± 0.060 %
Same top p: 87.894 ± 0.086 %