Week 11: Analyzing Fantasy QB Projections Performance

by thesanction1

I’ll keep today’s post very simple and lean more towards the anecdotal and philosophical than the analytical and concrete.  In future weeks I’d like to have more analytical information on how the picks actually performed, but that requires my computer not to be crunching the numbers needed to produce next weeks projections.  In short – I need additional computational resources to code up some quasi-relevant visualizations… If anyone makes a bunch of cash acting on my picks (not advised) and feels the compelling urge to ship me the new Mac Pro for Christmas, don’t hesitate!  The fantasy Gods will favor you indefinitely, and I wholeheartedly promise to be more prompt with my analysis. Anyway – in leu of going into detail about last weeks QB projections, which I would classify on the whole as only mildly successful, I’d like to discuss the merits of projections versus rankings, and go over a few of the key player projections compared against rankings of the ESPN fantasy experts for week 11.

Fantasy rankings vs. projections

Anyone who plays fantasy sports is familiar with the weekly ranking system. Fantasy experts across the industry project the rank order for their preference of players at each fantasy position for the upcoming week. Some hallmark examples of this are the aforementioned ESPN.com’s fantasy rankings, or those by the guys at CBS sports. The purpose of these rankings is to assist fantasy team managers when making decisions about which players to start in the coming week and help them decide which players on the waiver wire they ought to acquire.

The virtues of this ranking system are simple: it’s easy to understand, and it’s used almost everywhere. Rank ordering is something humans do almost compulsively – we love to know who was first place, and will do anything to avoid being last.  But what ranking systems provide in simplicity they lack in accuracy and information richness.  When I was getting my undergraduate in Physics I had a professor who excelled at breaking complex subjects down to make them more digestible. One of his favorite quotes was something he heard from the physics legend Richard Feynman at the 131st Gathering of Wizardly Nerds (not a real thing) about the challenges of creating any new theory about the world: “In any description of the natural world, there is always a tradeoff between clarity and correctness.” To stick with my longstanding (read: three-post) tradition of hyperbole on this blog, I’ll say that there’s no domain in which Dr. Feynman’s statements hold more true than in fantasy football.

Let’s look briefly at the follow of rankings in a domain that might be less laden with preconceptions about the rank ordering of things – global economics.  If I ask you which economies had the top 5 largest GDP’s in 2012 you might correctly list out the United States, China, Japan, Germany and France.  Well done!  Now let’s say I’ve tasked you with putting rostering a Fantasy Economics team, where the goal is to put together the team of 3 countries who will have the largest combined GDP for the 2013 season.  Taking my queues from the burgeoning daily fantasy sports industry, I’ve assigned salaries to each economy and you, as the coach, have a salary cap of $10,000 to spend on your team.  Let’s say the salaries are as follows:

United States  – $6,000

China  –  $4,000

Japan –  $3,000

Germany  –  $2,000

France  –  $1,000

What do you do?  You know that these are the top 5 economies from last year, and by all reasonable expectations they will be the top 5 economies for 2013.  If rank order is all you have to go on, and you avoid falling victim to your non-Commie biases, you will probably end up on a team consisting of  China, Japan, and Germany – giving you an average rank for members of your team of 3 (calculation: 1/3 * (2 + 3 + 4)), which is the highest possible average ranking for this scenario. The second highest ranking alternative that still fits in your $10,000 budget is a team of the United States, Germany, and France which has a slightly worse average rank of 3.333 (calculation:  1/3 * (1 + 4 + 5)). If all you had to go on were a ranking scale, you’d never rationally choose to play the second, inferior-ranked team.

Now let’s analyze the situation with the same salaries and salary cap, but this time with the actual GDP’s to help us make our decision instead of the rankings. Here were the actual GDP’s in 2012:

United States  –  $16.4 trillion

China  –  $7.3 trillion

Japan  –  $5.9 trillion

Germany  –  $3.4 trillion

France  –  $2.6 trillion

Again – we assume the GDP’s in 2013 to be roughly the same as last year.  Now how would you draft your team?  To avoid spontaneous brain melt, and to usher us more quickly to the point I’ll do the math… The optimal team you can afford with a $10,000 budget is now a team made up of the United States, Germany, and France – coming in at a combined total of $22.4 trillion (hoo-rahhh!).  What about our previously selected team with the higher average rank: China, Japan, and Germany?  They come in at an underwhelming total of $16.6 trillion.  If you built your Fantasy Economics team based on rankings alone, you got crushed.  In fact, armed with the actual figures it becomes clear that any combination of 3 countries that does not contain the United States has already lost.  I say this with some national pride but also to illustrate the critical point: rankings might tell us ‘who is better than who’, but they don’t tell us ‘by how much’.  Rankings are clearer, but projections are more correct.

The deficiency becomes even more glaring when you consider that fantasy owners are usually faced with upwards of 1,000 different players to chose from.  It would be critical to know whether or not daily fantasy, like fantasy economics, is a game where if you don’t have Drew Brees on your team you’re S.O.L.  With weekly projections you can come much closer to understanding the distribution of points scored for the week, and where to expect each player to fall in that distribution, instead of just rank preference. This is the holy grail of fantasy intelligence.

Anecdotes from week 11

Last week every analyst on ESPN.com ranked Drew Brees #1, and Peyton Manning #2 for fantasy quarterbacks.  My linear method also projected Brees as the #1 quarterback with 24.25 points, but projected Manning as the #11 ranked quarterback with a pedestrian 18.01 points behind guys like Ben Roethlisberger, Cam Newton, Matthew Stafford and Russell Wilson.  To me it seemed unlikely – but who was I to argue with a 15-dimensional linear regression model?

As is almost always the case in fantasy analysis – everyone was wrong, including the model.  Brees ended up scoring two fewer fantasy points than Manning, and neither one was a top 5 quarterback for the week.  Interestingly Manning was ranked #11 and projected at 18.01 points (he actually ended up the 13th ranked QB, putting up 20 points). My projections put him only .75 points below the projected point total for Alex Smith (projected at 18.76) who had the 6th highest projected score for fantasy quarterbacks (Smith actually ended up the 7th highest scoring quarterback, putting up 24.7 points).  This surprised the heck out of me, and probably the ESPN guys as well whose average ranking had Smith as 17th for the week. Here is how the machine and ESPN (E-rank) faired in ranking (M-rank) and projecting (M-proj.) the top 5 fantasy QB’s for the week compared to their actuals (act.):

1) Big Ben – act: 35.6 | M-proj: 18.1 | M-rank: 10th | E-rank: 14th

2) Cam Newton – act: 28.7 | M-proj: 18.2 | M-rank: 9th | E-rank: 5th

3) Carson Palmer – act: 28.7 | M-proj: 12.0 | M-rank: 28th | E-rank: 20th

4) Mathhew Stafford – act: 28.5 | M-proj: 19.5 | M-rank: 4th | E-rank: 4th

5) Nick Foles – act: 25.6 | M-proj: 19.1  | M-rank: 5th   | E-rank: 7th

I need to fully analyze the results to get a more comprehensive comparison against the guys at ESPN, but I can say immediately that we were wrong in very different ways.  The machine picked many unlikely success stories – ranking the likes of Alex Smith, and Big Ben higher than Peyton Manning; a move no analyst would be crazy enough to pull.  But it also does lots of stupid things like rating injured players highly, or projecting guys that have been recently named starters to produce like backups.

By the way – in comparing against ESPN analysts I’m not intending to start a turf war… I listen to the Fantasy Focus podcast almost every day and in many ways they got me obsessed with all of this stuff to begin with.  It’s a sign of respect, from myself – an absolutely insignificant pion of the fantasy analysis world – considers ESPN the standard against which I should compare my analysis. I have nothing but genuine respect for the work they do. Long live The Weaseldome and down with Pod Vader… Though, unlike Mathhew and Nate, I do enjoy the ‘Ahhhh! My finger!’ drop from Stephania that he plays incessantly every time she mentions the, er, ‘G-word’ in her injury reports.


When looking at rankings, the difference between the 6th and 11th ranked player might seem like a huge jump – but if they are projected within 1 point of each other, the 11th ranked player might present a great value play for the week.  The reasons for making numerical projections are thus fourfold:

1) Projections backed by solid statists are likely to outperform projections  arrived at through intuition

2) When making trade-off decisions, having only rank-order projections does not produce optimal teams under normal real-world conditions (i.e. when the distribution of points is likely non-normal) 

3) Only a statistical, computational based approach (the ultimate product of which is a projection) can consider all possibilities by looking at every matchup, considering every injury, for every position, each week.  The work required for a human to do this is astounding.

The fourth little trifle is worth mentioning here, but not quite worthy of it’s own bullet point: a ranking system will not tell you anything about what to expect for the week en aggregate. For most fantasy purposes this doesn’t matter.  But in certain circumstances it can present an opportunity to make a little extra cash. As an example, sites like www.DraftDay.com often times host games in which the goal is to attain a target of 150 fantasy points, and you are only competing against yourself.  In these instances the aggregate performance can be as important to consider as the rankings.  Only actual projections can give you a sense of overall scale.

That’s all for now – check back Saturday for more musings as well as Sunday’s predictions.