====== Perplexity statistics ======
Mean PPL(Q)                   :  26.530303 ±   0.256822
Mean PPL(base)                :  24.931431 ±   0.241228
Cor(ln(PPL(Q)), ln(PPL(base))):  98.58%
Mean ln(PPL(Q)/PPL(base))     :   0.062158 ±   0.001634
Mean PPL(Q)/PPL(base)         :   1.064131 ±   0.001738
Mean PPL(Q)-PPL(base)         :   1.598873 ±   0.044813

====== KL divergence statistics ======
Mean    KLD:   0.110374 ±   0.000408
Maximum KLD:   4.856987
99.9%   KLD:   1.554827
99.0%   KLD:   0.706525
99.0%   KLD:   0.706525
Median  KLD:   0.063658
10.0%   KLD:   0.001721
 5.0%   KLD:   0.000315
 1.0%   KLD:   0.000020
Minimum KLD:  -0.000091

====== Token probability statistics ======
Mean    Δp: -1.459 ± 0.021 %
Maximum Δp: 90.908%
99.9%   Δp: 41.637%
99.0%   Δp: 20.507%
95.0%   Δp:  8.036%
90.0%   Δp:  3.551%
75.0%   Δp:  0.185%
Median  Δp: -0.025%
25.0%   Δp: -2.071%
10.0%   Δp: -9.115%
 5.0%   Δp: -15.397%
 1.0%   Δp: -30.740%
 0.1%   Δp: -54.239%
Minimum Δp: -95.795%
RMS Δp    :  8.130 ± 0.041 %
Same top p: 84.107 ± 0.095 %