Ongoing recruiting project
I've been working on an ongoing project to better explain the extent of the usefulness (or lack thereof) of the recruiting rankings from rivals and scout. I've always felt that people invest way too much stock in the rankings, but I have to admit the correlation is better than I thought it would be.
Here's a slight macro look at what I have for rivals so far, I thought it was pretty interesting:
![[image]](http://i50.tinypic.com/153395t.jpg)
Notes:
* Red markers are teams that have won national championships in the 2002-2009 time period
* Total win % = total wins from 2002 through 2009 divided by total games played
* Boise St, TCU, Utah, and Duke are the most egregious outliers
I think the best explanation of why the regression line straightens out in the middle is because that's where the top teams in the low conference and the worst teams in the big conferences start mixing their recruiting rankings. It might be worthwhile to run two separate studies for BCS conference teams and non BCS conference teams.
The next step is to look at scout, and then both scout and rivals combined.
Any suggestions on what else to look at?
cool chart
by HumanRobot 
, Cybertron, Sunday, June 13, 2010, 15:25 (5841 days ago) @ suave_andrew
So Cincy has less talent than BSU, Utah, and TCU and plays in a relative big boy conference?
Cincinnati was in Conference USA until recently
I don't recall off the top of my head which year they switched over, but I think their recruiting has picked up a bit since joining the Big East and getting more exposure. That chart uses data all the way back until 2002, so it may not be totally representative of today's actual picture.
oh, forgot about that
by HumanRobot , Cybertron, Sunday, June 13, 2010, 15:56 (5841 days ago) @ suave_andrew
They joined the Big East in 2005.
relative recruiting rankings since 2005
by HumanRobot , Cybertron, Sunday, June 13, 2010, 17:40 (5841 days ago) @ HumanRobot
Rankings from Rivals
2004
TCU- 55
Utah- 61
BSU- 85
Cincy- 80
2005
TCU- 54
Utah- 59
BSU- 64
Cincy- 94
2006
TCU- 60
Utah- 55
BSU- 70
Cincy- 108
2007
TCU- 80
Utah- 71
BSU- 68
Cincy- 89
2008
TCU- 96
Utah- 60
BSU- 89
Cincy- 67
2009
TCU- 46
Utah- 44
BSU- 72
Cincy- 60
2010
TCU- 46
Utah- 32
BSU- 82
Cincy- 59
So it looks to me that BSU and Cincy have been somewhat comparable. Notably, Cincy's recruiting picked up about 20 spots with Kelly.
The more that I look at it, though, I think Suave's curve is interesting, when you look at x-axis positioning, you see that a lot of teams that Cincy tends to play are to the left. Look at last year's scheudle: Rutgers (~45), SEMS (??), OrSU (~45), FrSU (~75), USF (~50), Lou (~50), Syracuse (~60), UConn (~80), WVU (~35), Ill (~35), Pitt (~30). None of those dots are to the right of Cincy's average recruiting. Compare that to BSU's schedule: Oregon (~25), Miami OH (~90), FrSU (~75), BG (~95), UC-Davis (??), Tulsa (~75), Hawaii (~85), SJS (~95), Lou Tech (~85), Idaho (~95), Utah State (~95), Nevada (~85), and NMS (~110) and you start to get a picture that BSU tends to play teams closer to it in talent level, if not superior.
Again, though, great chart of Suave!
Very interesting.
by Jim (fisherj08) , A Samoan kid's laptop, Sunday, June 13, 2010, 14:43 (5841 days ago) @ suave_andrew
I would argue that the bigger thing at play is that recruiting rankings are good and ranking kids from, say, 1-100. After that, though, its a total crapshoot.
More anecdotal than what you're looking for
by CW (Rakes) , Harlan County, Sunday, June 13, 2010, 14:07 (5841 days ago) @ suave_andrew
...but it does confirm what your data is showing.
BGS, Dr. Saturday and MGo all did pieces where they went back over Top 100's from four or five years ago and looked at the success rate. More often than not the ranking services were very accurate in their projections, especially if you factor out things that would be more difficult to account for than just talent (injury, academics, headcases). Some quick Googling has yielded me nothing, but I'll keep poking around.
I recall the ones posted on BGS
They were interesting reads.
It's not that I don't think the recruiting services are useful, it's rather that I feel people place way too much significance in how meaningful the rankings are as a predictive tool of a team's performance.
The rankings do indeed hold SOME predictive power, more so than I thought, but the real key is trying to figure out a way to use them more productively than just saying "ND's average recruiting rankings for the last 4 years means we're the 10th most talented team this year!"
I'm going to eventually try and look at how to combine them with team experience, among other things. I've tried strength of schedule using recruiting rankings before and it only loosely works. It would be interesting to try and figure out some way to track how well coaches can develop talent, but that's way down the road.
It may be useful to look at the record against top 30 teams
Just winning % may be a bad metric given the weaknesses you cited. It is affected by who they play. BSU may be recruiting better than every team in the WAC which could explain their strong winning % but it is hardly indicative of a status as an elite team.
Scheduling always throws a monkey wrench into this
which is one of the reasons why I think it may be worth splitting recruiting rankings into BCS schools and non-BCS schools to help account for the scheduling/conference differences. At least until a better option comes around.
I like to use power ratings
by Jeff (BGS) , A starter home in suburban Tempe, Sunday, June 13, 2010, 16:42 (5841 days ago) @ suave_andrew
I'm partial to Sagarin, but there are plenty of good ones out there. One think I like is that differences between teams aren't as pronounced. The difference between 8-4 and 7-5 (or any two 12 game records) is something like 8%, but in reality the teams may only off by 1-2%.