| ====== Perplexity statistics ====== | |
| Mean PPL(Q) : 25.889366 ± 0.253265 | |
| Mean PPL(base) : 24.931431 ± 0.241228 | |
| Cor(ln(PPL(Q)), ln(PPL(base))): 98.77% | |
| Mean ln(PPL(Q)/PPL(base)) : 0.037703 ± 0.001531 | |
| Mean PPL(Q)/PPL(base) : 1.038423 ± 0.001590 | |
| Mean PPL(Q)-PPL(base) : 0.957936 ± 0.040635 | |
| ====== KL divergence statistics ====== | |
| Mean KLD: 0.091519 ± 0.000361 | |
| Maximum KLD: 8.082105 | |
| 99.9% KLD: 1.367977 | |
| 99.0% KLD: 0.621550 | |
| 99.0% KLD: 0.621550 | |
| Median KLD: 0.049987 | |
| 10.0% KLD: 0.001159 | |
| 5.0% KLD: 0.000199 | |
| 1.0% KLD: 0.000009 | |
| Minimum KLD: -0.000175 | |
| ====== Token probability statistics ====== | |
| Mean Δp: -0.368 ± 0.019 % | |
| Maximum Δp: 86.561% | |
| 99.9% Δp: 43.423% | |
| 99.0% Δp: 22.085% | |
| 95.0% Δp: 9.741% | |
| 90.0% Δp: 5.056% | |
| 75.0% Δp: 0.582% | |
| Median Δp: -0.001% | |
| 25.0% Δp: -0.958% | |
| 10.0% Δp: -6.146% | |
| 5.0% Δp: -11.492% | |
| 1.0% Δp: -26.685% | |
| 0.1% Δp: -50.676% | |
| Minimum Δp: -89.036% | |
| RMS Δp : 7.422 ± 0.040 % | |
| Same top p: 85.436 ± 0.091 % | |