https://www.sbnation.com/nfl/2018/4...backs-josh-allen-sam-darnold-projections-hype By Bill Connelly According to Football Outsiders’ DYAR ratings — defense-adjusted yards above replacement — the top five NFL quarterbacks in 2017 were New England’s Tom Brady, the Los Angeles Chargers’ Philip Rivers, New Orleans’ Drew Brees, Minnesota’s Case Keenum, and Pittsburgh’s Ben Roethlisberger. Brady was a moderately successful two-year starter at Michigan and a sixth-round draft pick. Rivers was a record-setting four-year starter at NC State and a top-five pick. Brees was an undersized but prolific three-year starter out of an early version of the spread offense. He was picked in the second round. Keenum was an undrafted journeyman and former two-star recruit who threw for nearly 20,000 yards out of the spread in college but just bounced to his fourth team in six seasons in the NFL. Roethlisberger was a small-school star, a three-year college starter at Miami (Ohio), and a top-15 pick. Three power-conference players and two mid-major stars. Two top picks, a second-rounder, and two draft afterthoughts. Two former blue-chippers and three two- or three-star guys. Three players with perfect QB size and two undersized gunslingers. And lest you think experience was too much of a predictor here (since four of these five are up there in years), the No. 6 (Jared Goff) and 8 (Carson Wentz) QBs on the DYAR list were second-year guys, so that only goes so far as well. The top of this list of passers was low on indicators and high on symbolism. The simple truth: It’s really, really hard to identify good quarterbacks before they become good quarterbacks. Heading into the NFL Draft at the end of April, so much analysis always focuses on the idea of projection, on what a guy might be able to do, not necessarily what he’s done. This makes sense, of course, to a point — so much of success at any level is based on situation, scheme, and circumstance. The right coach, teammates, or system can make an immense difference, plus these guys haven’t faced NFL talent, with NFL coaching, before. Plenty of QBs with great college stats have bombed out in the pros, and plenty with merely good stats have thrived. NFL GMs can be forgiven for thinking that, once we get a kid in our system, it’s all gonna work out just fine. We can fix his flaws and maximize his talent. Stats will only tell you so much. They are, dare we say, for losers. If we look at the right stats, however, and do so from the right perspective, we can still get further down the road than we would get just relying on basic stats or the eye test. For instance, we definitively know a prospect’s ceiling: His college stats. It makes sense, right? Just as we don’t expect a blue-chip running back to average 12 yards per carry in college like he perhaps did in high school, a college back who averaged seven yards per carry in college probably isn’t going to do so in the NFL. And the odds of a quarterback matching his college stats at the next level are almost null. That is, he won’t match his rate stats. Keenum, for instance, isn’t throwing 50 times per game in the NFL like he did in his junior year at Houston, so his per-game yardage totals will be different. But things like completion rate, interception rate, etc., can be more telling. And success rate can be extremely telling. Success rate and IsoPPP (isolated points per play) have been go-to stats for a lot of my college analysis in recent years. Success rate is a common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not (the terms: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down). IsoPPP, meanwhile, looks at the magnitude of the successful plays in terms of expected points. As it turns out, the correlation between one’s success rate in college and in his first four years in the pros is around 0.272, better than other rate stats I experimented with. There are 38 quarterbacks who a) were drafted from an FBS school between 2010-17 and b) have thrown at least 300 passes in the NFL. This is not a huge group, and it overlooks players who either just entered the league, have battled injuries, or, of course, weren’t good enough to throw 300 passes in the league. Looking at college-to-pro results will always have limitations like that. We can still pretty clearly learn something from these 38 QBs. For starters, none of them exceeded their college success rate in their first four years in the pros*. * Why first four years? Because that’s generally how long a rookie contract lasts. If you pick a guy who’s going to need two or three years of grooming, you might lose him as soon as he becomes a viable player. Your college success rate is your ceiling. Almost the only guys who came the close to matching their college success rates were near the bottom: Brock Osweiler (45.4 percent success rate in college, 43.9 percent in first four NFL years) mostly sat the bench for three seasons, then parlayed a fourth-year surge into a lofty contract (and promptly fell apart). Jake Locker (40.0 percent / 38.3 percent) was the least efficient college QB in the sample and one of the least efficient in the pros. New Viking Kirk Cousins (48.5 percent / 46.1 percent), a fourth-round pick in 2012, also perhaps benefited from early-career bench time before thriving. We’ll see if his sparkly new contract ends up a better investment than Osweiler’s. Osweiler and Cousins got to sit and learn for a while before being sent into action. A lot of top draft picks, however, were given early playing time, both because of their potential and the fact that teams had invested top draft picks in them. For players like Jameis Winston (45.8 percent success rate in the pros), Cam Newton (42.6 percent in his first four years), and Marcus Mariota (43.4 percent), this has worked out pretty well. For others, like Gabbert (32.9 percent), Tim Tebow (35.8 percent), and, thus far, Mitchell Trubisky (34.0 percent), it has meant early exposure of all their flaws to opponents. Gabbert and Tebow never really recovered, but Goff (29.6 percent in his rookie year, 44.3 percent in his second) did. NFL: Pro Bowl-NFC vs AFC Jared Goff’s first season in the pros was a nightmare. His second ended in the Pro Bowl. Aaron Doster-USA TODAY Sports We don’t learn as much about the guys with great college efficiency as the ones with statistical deficiencies. Your success rate is going to sink as the degree of difficulty improves, and while the most efficient college quarterbacks have the best odds of pro efficiency, the variance is pretty high. But your ceiling is your ceiling, and even if this doesn’t say much about guys with obscene college stats, it says a ton about the Lockers and/or Blaine Gabberts of the world, the guys with mediocre stats and standout physical traits, the guys about whom scouts will say “Yeah, his stats aren’t that good, but I can fix him. Just look at that arm!” So what does this tell us about this year’s draft prospects? Less than amazing things. I’ve been very confused by the chatter about this being an amazing QB draft class. The buzz began before the 2017 season and continued despite Wyoming’s Josh Allen regressing drastically from a statistical perspective, UCLA’s Josh Rosen continuing to struggle with injuries, and USC’s Sam Darnold dealing with some turnoveritis. I have long suspected that this QB buzz has come in part because most of the truly best players in the draft play positions that don’t tend to warrant the top pick — running back (Saquon Barkley), offensive guard (Quenton Nelson), safety (Minkah Fitzpatrick, Derwin James), inside linebacker (Roquan Smith). That is to some degree understandable. But you have to add a lot of favorable context to these stats to convince yourself that this is even an above-average QB crop. Since the highest four-year pro success rate from any QB in this sample is 46.1 percent (from both Cousins and, thus far, Dak Prescott), and since 35 of 38 quarterbacks in our draft sample were at least three percentage points lower in the NFL than in college (most were much further away than that), let’s set an artificial bar at 49.1 percent. Those at or above that mark are the ones with Prescott-level early-career efficiency potential.[/IMG]
Where do this year’s 13 primary QB prospects land? 2018 QB prospects with a career success rate of 49.1 percent or higher: Baker Mayfield, Oklahoma (54.8 percent) Sam Darnold, USC (52.0 percent) Mason Rudolph, Oklahoma State (50.0 percent) Logan Woodside*, Toledo (49.5 percent) Of the 38 players in the NFL sample, only Winston (57.1 percent) and Bradford (55.4) had higher career success rates in college than Mayfield. Winston’s first three seasons in the NFL have generated a 45.8 percent success rate; Bradford battled injury and a porous offensive line, generating a 37.2 percent success rate in his first two years before rising to 42.3 percent, near the league average (which is usually between 42.5 and 43 percent), in his next two. Again, since both were drafted so high, there was no sitting — they threw a combined 1,181 passes in their respective rookie seasons. Mayfield and Darnold will potentially be thrust into action just as quickly. Rudolph and Woodside, perhaps less so. * Note: these are raw stats, unadjusted for opponent, and we don’t have enough of a sample of QBs from mid-major schools to know how much of a difference to expect from that jump. But the guys in the sample — Colin Kaepernick (47.3 percent in college, 42.1 in the NFL), Andy Dalton (49.5, 42.6), Derek Carr (47.1, 41.0), and Blake Bortles (51.6, 39.6) — have made the statistical transition about the same as the power-conference guys. NCAA Football: Oklahoma Pro Day Baker Mayfield’s college stats were otherworldly. Mark D. Smith-USA TODAY Sports Here are the 2018 QBs who came relatively close to that 49 percent mark: 2018 QB prospects with a career success rate within two percentage points of 49.1 percent: Riley Ferguson, Memphis (49.0 percent) Nick Stevens, Colorado State (48.9 percent) J.T. Barrett, Ohio State (48.6 percent) Luke Falk, Washington State (48.0 percent) Lamar Jackson, Louisville (47.4 percent) Mike White, WKU (47.4 percent) You can’t really get a conclusive read on prospects within this range, especially someone like Lamar Jackson, whose rushing ability is incredible. (I limited this look to just pass proficiency.) You can, however, draw some pretty stark, alarming conclusions about prospects in this range: 2018 QB prospects with a career success rate lower than 47.1 percent: Rosen (46.6 percent) Litton (45.2 percent) Allen (43.3 percent) Of the 38 QBs in our pro sample, only one (Osweiller) managed a league-average passing success rate in the NFL over his first four years after producing a college success rate this low. The only two QBs in the lower-efficiency range who were drafted in the first round, as Rosen and Allen will be: Gabbert and Locker. Not the greatest of role models. Of course, Rosen’s career numbers were dragged down by the simple fact that he played as a true freshman. Allen didn’t, nor did Gabbert. Rosen’s success rate improved over his three seasons, from 44.8 percent as a freshman in 2015, to 46.3 percent in 2016, to a perfectly solid 48.7 percent last year. So maybe he’s in the clear. Allen, however? If you’re likely to finish, at best, two to three percent below your college success rate, that means his ceiling is around 40.5 to 41 percent. That’s Ryan Mallett territory (40.8 percent). As a ceiling. Are we sure we’re willing to spend a top-five pick on a guy who might, with some good breaks, become Ryan Mallett? Maybe he goes on to become the outlier of outliers, as insisted on by every draft scout who watches him throw in shorts against no defenders. But what an incredible gamble it will be for whatever team inevitably picks him in the top 10. NCAA Football: Potato Bowl-Central Michigan vs Wyoming Josh Allen’s Wyoming stats were ... lacking. Brian Losness-USA TODAY Sports To add further context to these numbers, let’s run some basic projections. To do so, though, let’s talk a moment about explosiveness. Back in January, I began playing with what I call marginal efficiency and marginal explosiveness. Marginal Efficiency: the difference between a player’s success rate (passing, rushing, or receiving) or success rate allowed (for an individual defender) and the expected success rate of each play based on down, distance, and yard line. Marginal Explosiveness: the difference between a player’s IsoPPP (passing, rushing, or receiving) or IsoPPP allowed (for an individual defender) and the expected IsoPPP value of each play based on down, distance, and yard line. For offensive players, the larger the positive value, the better. For defensive players, it’s the opposite — the more negative, the better. In my 2018 college football preview series, I have been integrating marginal efficiency and marginal explosiveness into my player analysis, and it works pretty well. You can never truly isolate one player’s performance from others’ using play-by-play stats — there’s always extra context to address — but this can perhaps take us further down the road. In this case, it basically tells us that big plays don’t carry over to the pros. While the correlation between one’s marginal efficiency in college and the pros is about the same as success rate, the correlation for marginal explosiveness was much lower (0.099). That is to say, there’s almost no relationship. This was what I expected to see, both because of the inherent randomness of big plays and the fact that, because of fewer lopsided matchups and/or crippling errors, there are fewer big plays in the pros. But it means that we have to stick mostly to efficiency when attempting to make college-to-pro projections. That’s a differentiation we can’t make with more standard stats like yards per attempt. I ran a simple regression to see how these players’ college stats might translate to the pros. I included explosiveness below as a way to figure out who we might be getting a false impression of as much as anything. Players like Mayfield, Rudolph, Woodside, Ferguson, and Allen likely benefited more from a level of explosiveness that won’t carry over to the pros. Mayfield nailed the efficiency element of the routine as well, however. Allen, on the other hand... Since a common retort from Allen advocates has been that his supporting cast was terrible in 2017, and that this should negate his mostly awful stats from last fall, I included a projection based on his 2016-only stats as well. Using 2016 upgrades him from DeShone Kizer to Tim Tebow. Stats will never tell you everything about what a player can do. In this case, though, it tells you what certain players probably can’t. And in the case of Josh Allen, it would take a spectacular outlier performance — one that hasn’t happened this decade — to live up to the expectations of the top-five or top-10 pick it appears he will become.
The NFL is later to the game with analytics because it is more of a team sport than MLB and has more moving parts than NBA or NHL, but eventually someone is going to figure out how to evaluate these guys. Will it be right 100% of the time, no, and I doubt that it is in any other sport, but this is the future in sports scouting (as the article said, the correlation is only about 27%, which was the highest among all rates they reviewed). Let's forget Darnold and Rosen for a moment; I think the discussion in the future about Mayfield vs. Allen is going to change how we view these players. The guy with the measureables who can throw or the guy who doesn't look the part but passes the "excel" test. Of course, both could stink, or both could be great, which will not help the discussion at all.
I work with numbers all day long. In fact, the word "analyst" is in my job title. So I certainly understand the value of analytics but we're simply not there with it comes to football, especially when it comes to player evaluation. As you eluded, there are way too many variables in the game. By the way, Mayfield's offense and lack of exposure to certain throws is more of the concern than not looking the part.
I can see the problem with Connelly's analysis in these two paragraphs. As it turns out, the correlation between one’s success rate in college and in his first four years in the pros is around 0.272, better than other rate stats I experimented with. There are 38 quarterbacks who a) were drafted from an FBS school between 2010-17 and b) have thrown at least 300 passes in the NFL. This is not a huge group, and it overlooks players who either just entered the league, have battled injuries, or, of course, weren’t good enough to throw 300 passes in the league. Looking at college-to-pro results will always have limitations like that. His sample size is just 38, and to make matters worse he says he experimented with other rate stats. When you have a small sample, you can't experiment with a bunch of variables to see which one provides the best fit. If you do, you'll inevitably get one of the variables to fit the sample fit by sheer random luck. And then when you apply this variable to a new sample, you get nonsensical projections that rank Woodside, Ferguson, and Stevens as better QBs than Rosen, Allen, and Jackson. Also, you can't omit QBs who didn't start in the NFL. Because when you do, you're implicitly assuming those QBs are as good as the average QB who did start in the NFL, which is clearly not true. So you either have to use a different type of regression that can account for QBs who didn't play or limit your analysis to QBs drafted in the first couple rounds.
Making decisions based on field play worked 100% of the time historically, correct? I think the point is to obtain and parse as much data as possible in conjunction with field play to make the most educated guess you can. That being said, I would absolutely agree with this theory - collegiate success (statistically) is the NFL ceiling for the vast majority of QBs.
yup and a guy like bryce petty who was good in college (62.3% 62TDs to 10INTs) but slated as a 4th round pick due to watching film, has onyl thrown 245 passes in the NFL so he wouldn't qualify as someone with good college stats but we all know he sucks balls and couldn't hit the broad side of a barn. watching him play those final games last year was brutal.
Baker M. Wins again..at least projected to have the best shot along with Sam D...and we could snag him!!!
Ask and you shall receive! Here's Petty's QBase as discussed on GGN: https://www.ganggreennation.com/2015/5/5/8557615/bryce-petty-and-qbase "Bryce Petty Mean Projection (Years 3-5 of career): -292 DYAR Odds: Bust: 80.2% Adequate Starter: 13.7% Upper Tier: 5.2% Elite: 0.9% Petty projects to be substantially worse than replacement level, in large part because QBASE questions the opposition that he faced in 2014. Petty accumulated his college stats against the 70th-toughest slate of opposing defenses. His 6.1 percent chance of developing into an upper-tier quarterback makes Petty unworthy of a third-round selection."
Here's Hackenberg: http://www.nydailynews.com/sports/f...enberg-sacked-nfl-analytics-article-1.2640197 "Hackenberg did not fare well. QBASE projects an 80.1% chance of him turning out to be a bust, with a mean projection total of -414 DYAR (defense-adjusted yards above replacement) in year’s 3-5 of his career. That rate, of -138 DYAR per year, is comparable to 2015 performances by Johnny Manziel, Matt Cassell and Andrew Luck last season."
Analytics is more predictive/accurate in other sports because of sample size. It’s how to draw deductions on 12 college games especially when supporting cast and opponents are not the same.
This is very true. Just thinking about how there's 155 Mets games left to play. (Still pleased with the start.) There's still a very, very long way to go to quantifying football game data in a usage fashion. Considering how behind the football (as an institution) has been, they're coming around. Slowly. (When you break down something like QBase, it doesn't capture any new expression of game data as opposed to contextualizing existing stats. Which is to say, its a stat of stats. Still, it's innovative for football. And progress is still progress.
