According to Football Outsiders’ DYAR ratings — defense-adjusted yards above replacement — the top five NFL quarterbacks in 2017 were New England’s Tom Brady, the Los Angeles Chargers’ Philip Rivers, New Orleans’ Drew Brees, Minnesota’s Case Keenum, and Pittsburgh’s Ben Roethlisberger.
- Brady was a moderately successful two-year starter at Michigan and a sixth-round draft pick.
- Rivers was a record-setting four-year starter at NC State and a top-five pick.
- Brees was an undersized but prolific three-year starter out of an early version of the spread offense. He was picked in the second round.
- Keenum was an undrafted journeyman and former two-star recruit who threw for nearly 20,000 yards out of the spread in college but just bounced to his fourth team in six seasons in the NFL.
- Roethlisberger was a small-school star, a three-year college starter at Miami (Ohio), and a top-15 pick.
Three power-conference players and two mid-major stars. Two top picks, a second-rounder, and two draft afterthoughts. Two former blue-chippers and three two- or three-star guys. Three players with perfect QB size and two undersized gunslingers. And lest you think experience was too much of a predictor here (since four of these five are up there in years), the No. 6 (Jared Goff) and 8 (Carson Wentz) QBs on the DYAR list were second-year guys, so that only goes so far as well.
Heading into the NFL Draft at the end of April, so much analysis always focuses on the idea of projection, on what a guy might be able to do, not necessarily what he’s done.
This makes sense, of course, to a point — so much of success at any level is based on situation, scheme, and circumstance. The right coach, teammates, or system can make an immense difference, plus these guys haven’t faced NFL talent, with NFL coaching, before. Plenty of QBs with great college stats have bombed out in the pros, and plenty with merely good stats have thrived.
NFL GMs can be forgiven for thinking that, once we get a kid in our system, it’s all gonna work out just fine. We can fix his flaws and maximize his talent. Stats will only tell you so much. They are, dare we say, for losers.
If we look at the right stats, however, and do so from the right perspective, we can still get further down the road than we would get just relying on basic stats or the eye test.
It makes sense, right? Just as we don’t expect a blue-chip running back to average 12 yards per carry in college like he perhaps did in high school, a college back who averaged seven yards per carry in college probably isn’t going to do so in the NFL. And the odds of a quarterback matching his college stats at the next level are almost null.
That is, he won’t match his rate stats. Keenum, for instance, isn’t throwing 50 times per game in the NFL like he did in his junior year at Houston, so his per-game yardage totals will be different. But things like completion rate, interception rate, etc., can be more telling. And success rate can be extremely telling.
Success rate and IsoPPP (isolated points per play) have been go-to stats for a lot of my college analysis in recent years.
- Success rate is a common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not (the terms: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down).
- IsoPPP, meanwhile, looks at the magnitude of the successful plays in terms of expected points.
As it turns out, the correlation between one’s success rate in college and in his first four years in the pros is around 0.272, better than other rate stats I experimented with.
There are 38 quarterbacks who a) were drafted from an FBS school between 2010-17 and b) have thrown at least 300 passes in the NFL. This is not a huge group, and it overlooks players who either just entered the league, have battled injuries, or, of course, weren’t good enough to throw 300 passes in the league. Looking at college-to-pro results will always have limitations like that.
We can still pretty clearly learn something from these 38 QBs. For starters, none of them exceeded their college success rate in their first four years in the pros*.
* Why first four years? Because that’s generally how long a rookie contract lasts. If you pick a guy who’s going to need two or three years of grooming, you might lose him as soon as he becomes a viable player.
Your college success rate is your ceiling.
Almost the only guys who came the close to matching their college success rates were near the bottom:
- Brock Osweiler (45.4 percent success rate in college, 43.9 percent in first four NFL years) mostly sat the bench for three seasons, then parlayed a fourth-year surge into a lofty contract (and promptly fell apart).
- Jake Locker (40.0 percent / 38.3 percent) was the least efficient college QB in the sample and one of the least efficient in the pros.
- New Viking Kirk Cousins (48.5 percent / 46.1 percent), a fourth-round pick in 2012, also perhaps benefited from early-career bench time before thriving. We’ll see if his sparkly new contract ends up a better investment than Osweiler’s.
Osweiler and Cousins got to sit and learn for a while before being sent into action. A lot of top draft picks, however, were given early playing time, both because of their potential and the fact that teams had invested top draft picks in them. For players like Jameis Winston (45.8 percent success rate in the pros), Cam Newton (42.6 percent in his first four years), and Marcus Mariota (43.4 percent), this has worked out pretty well.
For others, like Gabbert (32.9 percent), Tim Tebow (35.8 percent), and, thus far, Mitchell Trubisky (34.0 percent), it has meant early exposure of all their flaws to opponents. Gabbert and Tebow never really recovered, but Goff (29.6 percent in his rookie year, 44.3 percent in his second) did.
We don’t learn as much about the guys with great college efficiency as the ones with statistical deficiencies. Your success rate is going to sink as the degree of difficulty improves, and while the most efficient college quarterbacks have the best odds of pro efficiency, the variance is pretty high.
But your ceiling is your ceiling, and even if this doesn’t say much about guys with obscene college stats, it says a ton about the Lockers and/or Blaine Gabberts of the world, the guys with mediocre stats and standout physical traits, the guys about whom scouts will say “Yeah, his stats aren’t that good, but I can fix him. Just look at that arm!”
I’ve been very confused by the chatter about this being an amazing QB draft class. The buzz began before the 2017 season and continued despite Wyoming’s Josh Allen regressing drastically from a statistical perspective, UCLA’s Josh Rosen continuing to struggle with injuries, and USC’s Sam Darnold dealing with some turnoveritis.
I have long suspected that this QB buzz has come in part because most of the truly best players in the draft play positions that don’t tend to warrant the top pick — running back (Saquon Barkley), offensive guard (Quenton Nelson), safety (Minkah Fitzpatrick, Derwin James), inside linebacker (Roquan Smith).
That is to some degree understandable. But you have to add a lot of favorable context to these stats to convince yourself that this is even an above-average QB crop.
Since the highest four-year pro success rate from any QB in this sample is 46.1 percent (from both Cousins and, thus far, Dak Prescott), and since 35 of 38 quarterbacks in our draft sample were at least three percentage points lower in the NFL than in college (most were much further away than that), let’s set an artificial bar at 49.1 percent. Those at or above that mark are the ones with Prescott-level early-career efficiency potential.
Where do this year’s 13 primary QB prospects land?
2018 QB prospects with a career success rate of 49.1 percent or higher:
- Baker Mayfield, Oklahoma (54.8 percent)
- Sam Darnold, USC (52.0 percent)
- Mason Rudolph, Oklahoma State (50.0 percent)
- Logan Woodside*, Toledo (49.5 percent)
Of the 38 players in the NFL sample, only Winston (57.1 percent) and Bradford (55.4) had higher career success rates in college than Mayfield. Winston’s first three seasons in the NFL have generated a 45.8 percent success rate; Bradford battled injury and a porous offensive line, generating a 37.2 percent success rate in his first two years before rising to 42.3 percent, near the league average (which is usually between 42.5 and 43 percent), in his next two.
Again, since both were drafted so high, there was no sitting — they threw a combined 1,181 passes in their respective rookie seasons. Mayfield and Darnold will potentially be thrust into action just as quickly. Rudolph and Woodside, perhaps less so.
* Note: these are raw stats, unadjusted for opponent, and we don’t have enough of a sample of QBs from mid-major schools to know how much of a difference to expect from that jump. But the guys in the sample — Colin Kaepernick (47.3 percent in college, 42.1 in the NFL), Andy Dalton (49.5, 42.6), Derek Carr (47.1, 41.0), and Blake Bortles (51.6, 39.6) — have made the statistical transition about the same as the power-conference guys.
Here are the 2018 QBs who came relatively close to that 49 percent mark:
2018 QB prospects with a career success rate within two percentage points of 49.1 percent:
- Riley Ferguson, Memphis (49.0 percent)
- Nick Stevens, Colorado State (48.9 percent)
- J.T. Barrett, Ohio State (48.6 percent)
- Luke Falk, Washington State (48.0 percent)
- Lamar Jackson, Louisville (47.4 percent)
- Mike White, WKU (47.4 percent)
You can’t really get a conclusive read on prospects within this range, especially someone like Lamar Jackson, whose rushing ability is incredible. (I limited this look to just pass proficiency.)
You can, however, draw some pretty stark, alarming conclusions about prospects in this range:
2018 QB prospects with a career success rate lower than 47.1 percent:
- Rosen (46.6 percent)
- Litton (45.2 percent)
- Allen (43.3 percent)
Of the 38 QBs in our pro sample, only one (Osweiller) managed a league-average passing success rate in the NFL over his first four years after producing a college success rate this low. The only two QBs in the lower-efficiency range who were drafted in the first round, as Rosen and Allen will be: Gabbert and Locker. Not the greatest of role models.
Of course, Rosen’s career numbers were dragged down by the simple fact that he played as a true freshman. Allen didn’t, nor did Gabbert. Rosen’s success rate improved over his three seasons, from 44.8 percent as a freshman in 2015, to 46.3 percent in 2016, to a perfectly solid 48.7 percent last year. So maybe he’s in the clear.
Allen, however? If you’re likely to finish, at best, two to three percent below your college success rate, that means his ceiling is around 40.5 to 41 percent. That’s Ryan Mallett territory (40.8 percent). As a ceiling. Are we sure we’re willing to spend a top-five pick on a guy who might, with some good breaks, become Ryan Mallett?
Maybe he goes on to become the outlier of outliers, as insisted on by every draft scout who watches him throw in shorts against no defenders. But what an incredible gamble it will be for whatever team inevitably picks him in the top 10.
To add further context to these numbers, let’s run some basic projections. To do so, though, let’s talk a moment about explosiveness.
Back in January, I began playing with what I call marginal efficiency and marginal explosiveness.
Marginal Efficiency: the difference between a player’s success rate (passing, rushing, or receiving) or success rate allowed (for an individual defender) and the expected success rate of each play based on down, distance, and yard line.
Marginal Explosiveness: the difference between a player’s IsoPPP (passing, rushing, or receiving) or IsoPPP allowed (for an individual defender) and the expected IsoPPP value of each play based on down, distance, and yard line.
For offensive players, the larger the positive value, the better. For defensive players, it’s the opposite — the more negative, the better.
In my 2018 college football preview series, I have been integrating marginal efficiency and marginal explosiveness into my player analysis, and it works pretty well. You can never truly isolate one player’s performance from others’ using play-by-play stats — there’s always extra context to address — but this can perhaps take us further down the road. In this case, it basically tells us that big plays don’t carry over to the pros.
While the correlation between one’s marginal efficiency in college and the pros is about the same as success rate, the correlation for marginal explosiveness was much lower (0.099). That is to say, there’s almost no relationship.
This was what I expected to see, both because of the inherent randomness of big plays and the fact that, because of fewer lopsided matchups and/or crippling errors, there are fewer big plays in the pros. But it means that we have to stick mostly to efficiency when attempting to make college-to-pro projections. That’s a differentiation we can’t make with more standard stats like yards per attempt.
I ran a simple regression to see how these players’ college stats might translate to the pros. I included explosiveness below as a way to figure out who we might be getting a false impression of as much as anything.
Players like Mayfield, Rudolph, Woodside, Ferguson, and Allen likely benefited more from a level of explosiveness that won’t carry over to the pros. Mayfield nailed the efficiency element of the routine as well, however. Allen, on the other hand…
Since a common retort from Allen advocates has been that his supporting cast was terrible in 2017, and that this should negate his mostly awful stats from last fall, I included a projection based on his 2016-only stats as well.
Using 2016 upgrades him from DeShone Kizer to Tim Tebow.
And in the case of Josh Allen, it would take a spectacular outlier performance — one that hasn’t happened this decade — to live up to the expectations of the top-five or top-10 pick it appears he will become.