| ====== Perplexity statistics ====== | |
| Mean PPL(Q) : 8.376969 ± 0.060007 | |
| Mean PPL(base) : 7.669212 ± 0.052592 | |
| Cor(ln(PPL(Q)), ln(PPL(base))): 98.61% | |
| Mean ln(PPL(Q)/PPL(base)) : 0.088272 ± 0.001209 | |
| Mean PPL(Q)/PPL(base) : 1.092285 ± 0.001321 | |
| Mean PPL(Q)-PPL(base) : 0.707757 ± 0.011955 | |
| ====== KL divergence statistics ====== | |
| Mean KLD: 0.068549 ± 0.000337 | |
| Maximum KLD: 7.157206 | |
| 99.9% KLD: 1.521417 | |
| 99.0% KLD: 0.580272 | |
| 99.0% KLD: 0.580272 | |
| Median KLD: 0.033173 | |
| 10.0% KLD: 0.000904 | |
| 5.0% KLD: 0.000207 | |
| 1.0% KLD: 0.000022 | |
| Minimum KLD: -0.000013 | |
| ====== Token probability statistics ====== | |
| Mean Δp: -0.164 ± 0.018 % | |
| Maximum Δp: 91.952% | |
| 99.9% Δp: 40.903% | |
| 99.0% Δp: 20.124% | |
| 95.0% Δp: 9.817% | |
| 90.0% Δp: 5.860% | |
| 75.0% Δp: 1.292% | |
| Median Δp: -0.001% | |
| 25.0% Δp: -1.258% | |
| 10.0% Δp: -5.944% | |
| 5.0% Δp: -10.543% | |
| 1.0% Δp: -24.895% | |
| 0.1% Δp: -54.346% | |
| Minimum Δp: -93.879% | |
| RMS Δp : 7.202 ± 0.042 % | |
| Same top p: 88.757 ± 0.081 % | |