12-month international-entry backtest (monthly)

Validation selected base_rate by log loss. Parameters refit each January; inputs update at each forecast cutoff.

Test cutoffs: 2022-01-01 through 2024-12-01; 36 forecast dates. 1987 player snapshots; 93 players; 245 positive snapshots; 26 distinct observed entrants.

Labels are recorded entry and appearances. Neither a mature horizon nor more snapshots repairs archive gaps.

Model Log loss Brier Average precision ROC AUC Top-10 precision Mean probability
base_rate 0.384 0.110 0.111 0.437 18.4% 8.3%
boosting 0.339 0.097 0.366 0.756 33.1% 8.0%
boosting_context 0.334 0.096 0.386 0.771 32.2% 8.1%
context_only 0.384 0.110 0.121 0.472 18.4% 8.2%
logistic 0.345 0.093 0.417 0.735 34.4% 8.7%
logistic_context 0.342 0.093 0.415 0.743 32.8% 8.6%

Observed entry frequency: 12.3%. Lower log loss/Brier are better; higher ranking measures are better. Top ten are selected separately within each country/date; small pools can make recall mechanically perfect. Baseline ties use arbitrary UUID order.

Count model MAE Poisson deviance
mean_count 1.250 4.407
poisson 1.225 4.088
poisson_context 1.225 4.089
zero_count 0.856 inf

Counts measure opportunity and persistence. A zero forecast can win MAE but has infinite deviance on positive outcomes.

Country Snapshots Players Positive snapshots Unique entrants Max missing affiliation fixtures
England 1 1 1 1 1
India 1978 91 236 24 6
South Africa 8 1 8 1 2

Country is dated auction nationality, a proxy for affiliation. Outcomes and prior-cap exclusions span all national teams. The missing-fixture flag covers only the cutoff affiliation. Zero-coverage countries are excluded; partial coverage still limits labels.

Overlapping horizons, repeated players and country/calendar shocks make rows dependent. Improvements are descriptive backtest differences, not precise independent-row estimates. See equal-player metrics, year/country metrics, common-period scores and calibration CSVs. Overseas samples remain too small for broad validation.