24-month international-entry backtest (monthly)
Validation selected base_rate by log loss. Parameters refit each January; inputs update at each forecast cutoff.
Test cutoffs: 2022-01-01 through 2024-01-01; 25 forecast dates. 1358 player snapshots; 90 players; 295 positive snapshots; 25 distinct observed entrants.
Labels are recorded entry and appearances. Neither a mature horizon nor more snapshots repairs archive gaps.
| Model | Log loss | Brier | Average precision | ROC AUC | Top-10 precision | Mean probability |
|---|---|---|---|---|---|---|
| base_rate | 0.556 | 0.179 | 0.206 | 0.464 | 34.7% | 12.8% |
| boosting | 0.570 | 0.174 | 0.339 | 0.673 | 39.4% | 14.6% |
| boosting_context | 0.560 | 0.172 | 0.350 | 0.689 | 41.3% | 14.4% |
| context_only | 0.558 | 0.179 | 0.203 | 0.459 | 34.7% | 12.6% |
| logistic | 0.496 | 0.151 | 0.494 | 0.708 | 52.9% | 13.9% |
| logistic_context | 0.489 | 0.151 | 0.492 | 0.725 | 50.6% | 13.8% |
Observed entry frequency: 21.7%. Lower log loss/Brier are better; higher ranking measures are better. Top ten are selected separately within each country/date; small pools can make recall mechanically perfect. Baseline ties use arbitrary UUID order.
| Count model | MAE | Poisson deviance |
|---|---|---|
| mean_count | 3.167 | 9.755 |
| poisson | 3.016 | 8.634 |
| poisson_context | 3.016 | 8.634 |
| zero_count | 2.370 | inf |
Counts measure opportunity and persistence. A zero forecast can win MAE but has infinite deviance on positive outcomes.
| Country | Snapshots | Players | Positive snapshots | Unique entrants | Max missing affiliation fixtures |
|---|---|---|---|---|---|
| England | 1 | 1 | 1 | 1 | 2 |
| India | 1349 | 88 | 286 | 23 | 7 |
| South Africa | 8 | 1 | 8 | 1 | 6 |
Country is dated auction nationality, a proxy for affiliation. Outcomes and prior-cap exclusions span all national teams. The missing-fixture flag covers only the cutoff affiliation. Zero-coverage countries are excluded; partial coverage still limits labels.
Overlapping horizons, repeated players and country/calendar shocks make rows dependent. Improvements are descriptive backtest differences, not precise independent-row estimates. See equal-player metrics, year/country metrics, common-period scores and calibration CSVs. Overseas samples remain too small for broad validation.