The Value of QB’s in Daily vs Season-long Fantasy Football
Earl ‘Mitch’ Mitchell, StarsDraft.com’s Chief Information Officer, outlines below the StarsDraft.com decision to build a position agnostic player pricing model for our daily fantasy football platform offering. Mitch’s previous background as the Director of Technology at Wolfram Alpha’s predictive analytics consultancy arm helps lend a unique view and approach towards daily fantasy football — and all daily fantasy sports — when it comes to the day-in, day-out player pricing models.
I’ve played DFS for several years now and seen many DFS sites come and go (this is particularly true more recently). Each site has had its own unique take on the player pool and player pricing.
Some offered expansive player pools including irrelevant 5th string WR’s and backup K’s in an effort to make sure they didn’t leave any athletes out. Some had sharp salaries, some not so sharp. Some had tight caps, some loose. I have always been interested in what went on behind the scenes at these sites; what was the thought process behind salary/cap/player pool determinations? Were they trying to be predictive? Were they trying to anticipate user beliefs about the athletes to increase team diversity in large-field contests? Were they optimizing some other metric I hadn’t thought of? Were they just throwing darts?
Now I’ve stepped into the CIO role at StarsDraft.com and have to make those determinations myself. I had to think long and hard about what I felt the salaries setters should be doing – it’s not immediately clear. Hanging in the balance of these decisions is nearly every aspect of the user experience, from roster creation to the large field in-contest sweat. After weighing the plusses and minuses of different player pricing strategies I arrived at the following goal: salaries should be as sharp (i.e. predictive) as possible, and the cap should be set in such a way that the user is always forced to make tradeoffs when filling out their roster, but can still roster a few of the highest priced athletes (studs) if they are willing to roster some value (potentially deep value) plays.
Making salaries tighter or looser is easy – just lower or raise the salaries as a whole (or, tackling the problem the other way around, lower or raise the salary cap). But having sharp, predictive salaries is complicated – really complicated in fact. Do you have a single equation, in which the athletes position is just another variable? Do you have one equation per position? Per athlete?! What algorithm is best suited for this kind of problem? How often does it need to be retrained? How do you measure accuracy and avoid overfitting? How do you handle injury and other news that’s not mathematically tractable? Do you filter outliers? What counts as an outlier? And the list of concerns goes on, and on.
As I started building models, creating and testing different variables for predictiveness, trying different algorithms etc., one thing continued to crop up: the QB position was just much more valuable than the others. Consistently the top 5-10 players projected for each week would be QB’s. From my DFS experience I knew this just wasn’t how salaries worked, on a given week there may be 1-2 QB’s that were priced well beyond any other players (usually Drew Brees and Payton Manning), and then inevitably the 3rd highest priced athlete would be a RB (usually Adrian Peterson) followed by a mix of QB’s, more RB’s and somewhere in the top 10 Calvin Johnson and maybe one other stud WR. There’s always a RB or two in the top 5 and a WR in the top 10. But, according to my historical data explorations, this just wasn’t realistic.
I obviously can’t divulge the methods employed to create our salaries, but to demonstrate my point that QB’s are being underpriced in DFS I wont need to. Below is a chart of the top 15 scoring athletes (sorted by per-game-played-average) for the last 3 years at each of the big 4 fantasy positions: QB, RB, WR and TE.
Only once in the last three years was a RB snuck into the top 5 of average fantasy points per game. Only 3 RB’s have made the top 10 in as many years, and only 4 have made the top-15. The story is much, much worse for WR’s. In the last 3 years only 3 WR’s have cracked the top-20! in terms of average fantasy points per game. Keep in mind that these numbers are per-game-played, the calculation here doesn’t punish players who missed entire games due to injury (we are assuming DFS’ers are not playing an athlete if the athlete isn’t playing – this is harder to do than you might think). Obviously RB’s and WR’s are much more likely to be injured in a game than are QB’s, and much more likely to be game-time decisions that end up not playing. It’s no wonder the algorithms are pricing QB’s at a premium to the other positions – QB’s tend to put up substantially more fantasy points!
But wait… Anyone who plays season long leagues seriously knows that you shouldn’t draft a QB in the first round. And if you miss out on the big 3 (maybe big 4 if you include Stafford as some analysts are this year) you should likely just wait and pick up whoever is available in the later rounds. So if we don’t care about QB’s in season long, why should we care about them in DFS?
The answer is obvious – in season long leagues you are drafting from a pool of players, against other drafters, where your picks prevent other drafters from taking an athlete. With the typical roster you are drafting 2 RB’s, 2 WR’s, a TE and a FLEX – with 6 non-QB skill positions to fill, plus a bench to accommodate bye weeks and inevitable injuries, the advantage gained over the competition by having a consistent difference maker like Calvin Johnson is enormous. In daily fantasy football everyone can draft Calvin Johnson if they think he’s a good value for the week; we aren’t going to run out of quality RB/WR/TE’s. Another way to look at this is from the point-of-view of objectives. In daily fantasy roster creation you simply want to put together the team that will get you the most points in the coming week. That’s a very different objective from trying to outmaneuver your league mates in a snake draft to put together the best rosterthat can win week-in week-out for 17 weeks.
Alright, so QB’s should be valued a bit higher in DFS based on average output. But what about the second piece of any predictive puzzle: variance? Maybe it’s the case that QB’s average more points, but it is inherently unpredictable when they will have big weeks and when they will flop. Below is that same top-15 list, but this time sorted by their coeficient of variation – a measure of each athlete’s mean adjusted variation week-to-week:
Again – this data only includes games in which the athlete played. What does the above table show? Not only are RB’s and WR’s scoring fewer points on average, they are doing so with generally greater variance than the QB position. In terms of pricing this would imply that QB’s should be priced higher based on average output and the fact that we can better anticipate when they will do well compared to their RB, WR and TE counterparts.
Any set of salaries has the potential to cause a stir in the DFS community. Pricing QB’s predictively means that QB’s will cost a bit more and RB’s and WR’s will generally be more affordable. I have no idea how users will react to this shift. But the question should be answered or at least discussed: if salary makers aren’t making predictive salaries, then what are they doing? If the QB is the most productive (and most predictably productive!) position, shouldn’t we price it as such?