Backtest — walk-forward grid & 2026 holdout
Walk-forward folds predict each next-year finish from prior-season data only. The 2024-25 → 2026 holdout below uses hand-curated tier A ground truth. Top-20 MAE is the primary metric for scheduling relevance.
Walk-forward grid
24 folds · 5 full-model (2020+). Positive Δ top-20 MAE = roster beats naive (prior finish).
Full model — median Δ top-20
0.10
Roster wins 4/5 folds
Tier A ground truth — median Δ top-20
0.08
Roster wins 2/2 folds
Published holdout
2024-25 → 2026
Detail tables below
| Season | Forecast | Tier | GT | Naive all | Roster all | Δ all | Naive top-20 | Roster top-20 | Δ top-20 |
|---|---|---|---|---|---|---|---|---|---|
| 1999-00 | 2001 | results_rosters | B | 2.08 | 5.75 | -3.67 | 2.08 | 5.75 | -3.67 |
| 2000-01 | 2002 | results_rosters | B | 2.92 | 1.88 | 1.04 | 2.92 | 1.88 | 1.04 |
| 2001-02 | 2003 | results_rosters | B | 2.43 | 2.14 | 0.29 | 2.43 | 2.14 | 0.29 |
| 2002-03 | 2004 | results_rosters | B | 1.43 | 2.45 | -1.03 | 1.43 | 2.45 | -1.03 |
| 2003-04 | 2005 | results_rosters | B | 2.43 | 2.83 | -0.40 | 2.43 | 2.83 | -0.40 |
| 2004-05 | 2006 | results_rosters | B | 2.32 | 2.33 | -0.02 | 2.32 | 2.33 | -0.02 |
| 2005-06 | 2007 | results_rosters | B | 2.95 | 2.79 | 0.16 | 3.00 | 2.78 | 0.22 |
| 2006-07 | 2008 | results_rosters | B | 3.95 | 3.80 | 0.15 | 3.78 | 3.53 | 0.25 |
| 2007-08 | 2009 | results_rosters | B | 3.17 | 3.23 | -0.06 | 3.55 | 3.21 | 0.34 |
| 2008-09 | 2010 | results_rosters | B | 2.25 | 2.27 | -0.02 | 2.40 | 2.47 | -0.07 |
| 2009-10 | 2011 | results_rosters | B | 2.17 | 2.09 | 0.08 | 2.25 | 2.16 | 0.09 |
| 2010-11 | 2012 | results_rosters | B | 3.17 | 3.84 | -0.68 | 3.45 | 3.50 | -0.05 |
| 2011-12 | 2013 | results_rosters | B | 3.00 | 2.90 | 0.10 | 3.25 | 2.94 | 0.31 |
| 2012-13 | 2014 | results_rosters | B | 3.12 | 3.55 | -0.42 | 3.30 | 3.39 | -0.09 |
| 2013-14 | 2015 | results_rosters | B | 3.54 | 3.73 | -0.19 | 3.05 | 2.38 | 0.67 |
| 2014-15 | 2016 | results_rosters | B | 3.81 | 4.52 | -0.71 | 4.17 | 4.12 | 0.04 |
| 2015-16 | 2017 | results_rosters | B | 4.97 | 5.07 | -0.10 | 4.79 | 4.50 | 0.29 |
| 2016-17 | 2018 | results_rosters | B | 5.97 | 5.88 | 0.09 | 7.00 | 6.75 | 0.25 |
| 2017-18 | 2019 | results_rosters | B | 6.27 | 6.10 | 0.17 | 6.30 | 5.84 | 0.46 |
| 2020-21 | 2022 | full | B | 7.68 | 7.79 | -0.11 | 7.35 | 7.40 | -0.05 |
| 2021-22 | 2023 | full | B | 8.38 | 7.67 | 0.71 | 8.26 | 7.11 | 1.15 |
| 2022-23 | 2024 | full | B | 5.68 | 5.60 | 0.08 | 6.55 | 6.45 | 0.10 |
| 2023-24 | 2025 | full | A | 3.89 | 3.23 | 0.67 | 3.75 | 3.70 | 0.05 |
| 2024-25 | 2026 | full | A | 2.54 | 2.76 | -0.22 | 2.60 | 2.50 | 0.10 |
2024-25 holdout → predict 2026 finish
Same pipeline, but only with 2024-25 stats and 2025 graduates. Compare the prediction to
actual finish_2026.tsv.
Prior finish (2025)
2.54
overall MAE · 37 teams · within ±3 = 70%
Roster-adjusted (prior + turnover)
2.77
overall MAE · 34 teams · within ±3 = 65%
Roster-adjusted + secondary (board line)
2.77
overall MAE · 34 teams · within ±3 = 65%
Composite re-sort (legacy)
6.71
overall MAE · 34 teams · within ±3 = 50%
MAE by finish tier
Tier 1 = ranks 1-4, tier 2 = 5-8, tier 3 = 9-16, tier 4 = 17-32.
| Predictor | Tier 1 | Tier 2 | Tier 3 | Tier 4 |
|---|---|---|---|---|
| Composite re-sort (legacy) | 5.25 3/4 ±3 | 9.50 2/4 ±3 | 3.62 6/8 ±3 | 7.78 6/18 ±3 |
| Prior finish (2025) | 1.75 4/4 ±3 | 1.50 4/4 ±3 | 3.25 4/8 ±3 | 2.62 14/21 ±3 |
| Roster-adjusted (prior + turnover) | 1.75 4/4 ±3 | 1.25 4/4 ±3 | 3.12 3/8 ±3 | 3.17 11/18 ±3 |
| Roster-adjusted + secondary (board line) | 1.75 4/4 ±3 | 1.25 4/4 ±3 | 2.88 3/8 ±3 | 3.28 11/18 ±3 |
Per-team rank errors
| Team | Actual | Tier | Prior pred | Prior err | Roster pred | Roster err | Adjusted pred | Adjusted err | Composite pred | Composite err |
|---|---|---|---|---|---|---|---|---|---|---|
| Eureka | 1 | 1 | 3 | 2 | 3 | 2 | 3 | 2 | 3 | 2 |
| Ladue Horton Watkins | 2 | 1 | 4 | 2 | 4 | 2 | 4 | 2 | 18 | 16 |
| John Burroughs | 3 | 1 | 1 | -2 | 1 | -2 | 1 | -2 | 1 | -2 |
| Cor Jesu Academy | 4 | 1 | 5 | 1 | 5 | 1 | 5 | 1 | 5 | 1 |
| MICDS | 5 | 2 | 2 | -3 | 2 | -3 | 2 | -3 | 4 | -1 |
| St. Joseph's Academy | 6 | 2 | 6 | 0 | 6 | 0 | 6 | 0 | 21 | 15 |
| Visitation Academy | 7 | 2 | 10 | 3 | 8 | 1 | 8 | 1 | 6 | -1 |
| O'Fallon | 8 | 2 | 8 | 0 | 9 | 1 | 9 | 1 | 29 | 21 |
| Westminster | 9 | 3 | 7 | -2 | 7 | -2 | 7 | -2 | 8 | -1 |
| St. Dominic | 10 | 3 | 14 | 4 | 14 | 4 | 14 | 4 | 11 | 1 |
| Lafayette | 11 | 3 | 11 | 0 | 11 | 0 | 11 | 0 | 12 | 1 |
| Northwest | 12 | 3 | 16 | 4 | 16 | 4 | 16 | 4 | 15 | 3 |
| Villa Duchesne | 13 | 3 | 21 | 8 | 20 | 7 | 18 | 5 | 13 | 0 |
| Marquette | 14 | 3 | 9 | -5 | 10 | -4 | 10 | -4 | 17 | 3 |
| Wentzville | 15 | 3 | 15 | 0 | 15 | 0 | 15 | 0 | 26 | 11 |
| Rockwood Summit | 16 | 3 | 13 | -3 | 12 | -4 | 12 | -4 | 7 | -9 |
| Parkway West | 17 | 4 | 12 | -5 | 13 | -4 | 13 | -4 | 32 | 15 |
| Parkway South | 18 | 4 | 26 | 8 | 26 | 8 | 24 | 6 | 10 | -8 |
| Ursuline Academy | 19 | 4 | 19 | 0 | 19 | 0 | 20 | 1 | 30 | 11 |
| Webster Groves | 20 | 4 | 20 | 0 | 21 | 1 | 23 | 3 | 25 | 5 |
| Lindbergh | 21 | 4 | 24 | 3 | 25 | 4 | 26 | 5 | 22 | 1 |
| Nerinx Hall | 22 | 4 | 18 | -4 | 18 | -4 | 19 | -3 | 9 | -13 |
| Kirkwood | 23 | 4 | 22 | -1 | 22 | -1 | 22 | -1 | 16 | -7 |
| Francis Howell | 24 | 4 | 17 | -7 | 17 | -7 | 17 | -7 | 27 | 3 |
| Francis Howell North | 25 | 4 | 28 | 3 | 28 | 3 | 29 | 4 | 24 | -1 |
| Francis Howell Central | 26 | 4 | 25 | -1 | 24 | -2 | 25 | -1 | 20 | -6 |
| Clayton | 27 | 4 | 29 | 2 | 29 | 2 | 28 | 1 | 14 | -13 |
| Parkway Central | 28 | 4 | 23 | -5 | 23 | -5 | 21 | -7 | 2 | -26 |
| Fox High School | 29 | 4 | 33 | 4 | 31 | 2 | 31 | 2 | 19 | -10 |
| Oakville | 30 | 4 | 32 | 2 | 32 | 2 | 32 | 2 | 23 | -7 |
| Notre Dame | 31 | 4 | 27 | -4 | 27 | -4 | 27 | -4 | 31 | 0 |
| Incarnate Word | 32 | 4 | 30 | -2 | 30 | -2 | 30 | -2 | 34 | 2 |
| Parkway North | 33 | 4 | 31 | -2 | None | None | None | None | None | None |
| Pattonville | 34 | 4 | 34 | 0 | None | None | None | None | None | None |
| Whitfield | 35 | 4 | 35 | 0 | None | None | None | None | None | None |
| Edwardsville | 36 | 4 | 37 | 1 | 33 | -3 | 33 | -3 | 33 | -3 |
| Belleville West | 37 | 4 | 38 | 1 | 34 | -3 | 34 | -3 | 28 | -9 |