You're referring to them as "not reliable" but I'm not sure that's
what you're demonstrating.
What you're demonstrating is that the recruiting rankings lack predictive validity when (using NFL draft status as the outcome of interest).
Before deciding if recruiting rankings are "BS," one might want to consider a few relevant questions:
1. What are recruiting rankings and ratings intended to measure?
2. For what, if anything, are recruiting rankings and ratings appropriate for predicting?
Measures (and that is what recruiting rankings and ratings are) are generally not viewed as valid or invalid. Rather, a measure is viewed as valid or invalid FOR A PARTICULAR USE.
It would never occur to me to use the rankings of high school football players to try to predict future NFL draft position. The NFL draft has a extremely small base rate and there are so many intervening variables between high school and the draft. Therefore, the failure of recruiting rankings to predict NFL draft position doesn't, in my mind, invalidate recruiting rankings.
I would look to a more proximal criterion than NFL draft position. If I'm not mistaken, most recruiting services rate players and use the star system to describe a player's readiness to contribue at the college level (a more proximal outcome to high school football than the NFL draft).
Of course, there will be LOTS of other variables that also contribue to a player's readiness. S&C, avoidance of injuries, the system, depth and talent and one's position, the ability of the position coach to develop the player's talents, etc.
If you expect recruiting rankings to account for most of variance in college football performance, or more outlandishly, NFL draft position, then you have very unrealistic expectations for predictive models of performance outcomes.
If recruiting rankings can account for 10% of the variance in college player performance (that isn't accounted for by other predictors), that's an incremental gain in knowledge and prediction -- and therefore isn't BS.
Just My Opinion.
|
(
In response to this post by Flanker)
Posted: 12/22/2021 at 4:40PM