| ====== Perplexity statistics ====== | |
| Mean PPL(Q) : 26.552814 ± 0.258782 | |
| Mean PPL(base) : 24.931431 ± 0.241228 | |
| Cor(ln(PPL(Q)), ln(PPL(base))): 97.86% | |
| Mean ln(PPL(Q)/PPL(base)) : 0.063006 ± 0.002011 | |
| Mean PPL(Q)/PPL(base) : 1.065034 ± 0.002142 | |
| Mean PPL(Q)-PPL(base) : 1.621384 ± 0.054614 | |
| ====== KL divergence statistics ====== | |
| Mean KLD: 0.164859 ± 0.000610 | |
| Maximum KLD: 8.583980 | |
| 99.9% KLD: 2.315967 | |
| 99.0% KLD: 1.085407 | |
| 99.0% KLD: 1.085407 | |
| Median KLD: 0.093793 | |
| 10.0% KLD: 0.002759 | |
| 5.0% KLD: 0.000509 | |
| 1.0% KLD: 0.000027 | |
| Minimum KLD: -0.000053 | |
| ====== Token probability statistics ====== | |
| Mean Δp: -1.105 ± 0.025 % | |
| Maximum Δp: 95.542% | |
| 99.9% Δp: 53.440% | |
| 99.0% Δp: 27.342% | |
| 95.0% Δp: 11.606% | |
| 90.0% Δp: 5.754% | |
| 75.0% Δp: 0.494% | |
| Median Δp: -0.008% | |
| 25.0% Δp: -1.857% | |
| 10.0% Δp: -9.691% | |
| 5.0% Δp: -17.127% | |
| 1.0% Δp: -37.275% | |
| 0.1% Δp: -65.081% | |
| Minimum Δp: -94.606% | |
| RMS Δp : 9.854 ± 0.049 % | |
| Same top p: 80.956 ± 0.102 % | |