Monthly international-entry backtests
Predictions update monthly; model parameters refit each January using fully matured training labels. Primary horizon: 12 months. Evaluated horizons: 6, 12, 24 months. Model choice uses mature 2017–2019 validation outcomes.
Same-date comparison across horizons
All rows below use shared forecast dates from 2022-01-01 through 2024-01-01. Each horizon has a different outcome frequency: compare improvement over its own baseline, not raw scores between horizons.
| Horizon | Chosen model | Log loss | Base-rate log loss | Improvement | Brier | Average precision | Players | Unique entrants |
|---|---|---|---|---|---|---|---|---|
| 6 months | context_only | 0.270 | 0.270 | -0.1% | 0.069 | 0.112 | 90 | 20 |
| 12 months | base_rate | 0.437 | 0.437 | 0.0% | 0.128 | 0.147 | 90 | 25 |
| 24 months | base_rate | 0.556 | 0.556 | 0.0% | 0.179 | 0.206 | 90 | 25 |
Positive improvement means lower log loss. These are descriptive differences; repeated players and overlapping windows are not independent observations.
All 12-month classifiers
Every family was specified before the monthly backtest. The lowest test loss does not override validation model selection.
| Model | Validation log loss | Test log loss | Test Brier | Test average precision | Test ROC AUC | Test top-10 precision |
|---|---|---|---|---|---|---|
| base_rate | 0.287 | 0.384 | 0.110 | 0.111 | 0.437 | 18.4% |
| boosting | 0.360 | 0.339 | 0.097 | 0.366 | 0.756 | 33.1% |
| boosting_context | 0.369 | 0.334 | 0.096 | 0.386 | 0.771 | 32.2% |
| context_only | 0.287 | 0.384 | 0.110 | 0.121 | 0.472 | 18.4% |
| logistic | 0.348 | 0.345 | 0.093 | 0.417 | 0.735 | 34.4% |
| logistic_context | 0.345 | 0.342 | 0.093 | 0.415 | 0.743 | 32.8% |
Full 12-month test
The chosen base_rate model has log loss 0.384, Brier 0.110, average precision 0.111, ROC AUC 0.437, and top-10 precision 18.4%. Mean predicted probability: 8.3%; observed entry frequency: 12.3%. Sample: 1987 snapshots, 93 distinct players, 26 distinct observed entrants.
Top-ten rankings use country/month pools. Calibration bins, all model families, count forecasts, annual breakdowns, country breakdowns and equal-player metrics are in each run folder.
January-only benchmark
Both procedures below are evaluated on the same January candidate/date rows. Each chooses its model using its own validation cadence. The CSV also compares every identical model family.
| Training snapshots | Chosen model | January test log loss | Brier | Average precision |
|---|---|---|---|---|
| monthly | base_rate | 0.435 | 0.128 | 0.137 |
| annual | context_only | 0.428 | 0.126 | 0.216 |
This compares forecasting procedures, not a causal effect of more observations. Fixed regularization also behaves differently as training rows increase.
Horizon consistency and limitations
Independent horizon models violate increasing probability with longer horizons in 1/1358 shared forecasts. Raw probabilities are retained; they are not a coherent survival curve.
The sample still covers IPL pathways. Country affiliation is conservatively dated, international history is incomplete, and overseas cohorts are small. Outcomes describe recorded entry; this does not establish comprehensive cross-country accuracy or discrimination.
For definitions, censoring, leakage safeguards and design choices see monthly design.