====== Perplexity statistics ======
Mean PPL(Q)                   :  10.188365 ±   0.074008
Mean PPL(base)                :  10.036835 ±   0.072696
Cor(ln(PPL(Q)), ln(PPL(base))):  99.71%
Mean ln(PPL(Q)/PPL(base))     :   0.014985 ±   0.000549
Mean PPL(Q)/PPL(base)         :   1.015097 ±   0.000557
Mean PPL(Q)-PPL(base)         :   0.151531 ±   0.005697

====== KL divergence statistics ======
Mean    KLD:   0.016340 ±   0.000073
Maximum KLD:   1.379349
99.9%   KLD:   0.306994
99.0%   KLD:   0.123776
99.0%   KLD:   0.123776
Median  KLD:   0.009530
10.0%   KLD:   0.000284
 5.0%   KLD:   0.000056
 1.0%   KLD:   0.000003
Minimum KLD:  -0.000110

====== Token probability statistics ======
Mean    Δp: -0.221 ± 0.009 %
Maximum Δp: 58.518%
99.9%   Δp: 19.108%
99.0%   Δp:  9.095%
95.0%   Δp:  4.085%
90.0%   Δp:  2.282%
75.0%   Δp:  0.400%
Median  Δp: -0.002%
25.0%   Δp: -0.680%
10.0%   Δp: -2.932%
 5.0%   Δp: -5.028%
 1.0%   Δp: -11.387%
 0.1%   Δp: -25.096%
Minimum Δp: -63.543%
RMS Δp    :  3.306 ± 0.023 %
Same top p: 93.581 ± 0.064 %