Fantasy Football Fact or Fluke: When are sample sizes big enough to trust a player?

In science, it’s good practice to figure out how many experimental subjects you need to study in order to reach the conclusions you want to reach. Doing this work upfront ensures that you enroll the correct number of patients in a drug trial or college students in a study on how first-person-shooter video games improve visual perception (they do!).

Basically, it gives you the confidence to believe in the results you obtain after the experiment is done.

In fantasy sports, we would love to know how many games we must see to believe in who a player is. In theory, an effect-size analysis or time-series analysis would be able to provide this information, but the reality is much different. I’m not a statistician, but I am a fan of AI. Both Perplexity.ai and ChatGPT tell me that there are too many variables to make this kind of test applicable to fantasy sports. One of the benefits of laboratory research is the ability to control for a majority of factors while looking at how a single variable changes.

In football, we not only have a lot of different stats to consider as outcome measures (touchdowns, receiving yards, fantasy points) but every game is independent. These differences range from time of day, weather, opponent skill, internal state — sick, angry, worried, injured, motivated by specific (differing) narratives — to different game scripts, quarterback (or receivers), even different positional usage (such as a wide receiver operating as a running back).

If you pulled out only instances where all the factors were the same, you’d end up with an even smaller sample size that wouldn’t be applicable to most of the players’ games. It’s frustrating, but we are probably never going to get to a point where we can watch one or two of Keaton Mitchell’s games and tell that he’s going to be a top-five RB for the next four years.

To be honest, I think that’s part of what makes fantasy football enjoyable. Some managers need to see it ad nauseam to believe it — they pick up a hot player off waivers, but never feel good enough to start him fearing that it was a one-off, or maybe he had two good games because of matchup or game script but definitely can’t be trusted versus, say, Cleveland. Other managers live for the risk of riding the hot new hand, throwing the new pickup into their starting lineup immediately. They’re prepared to be disappointed, but don’t want to risk missing the upside of their new find if he does continue to shine.

The beauty is that both are right some of the time.

I’m going to touch on some players below who have flashed for better or worse in a limited number of games, some more than others. To disclose my personal bias, I tend to fall into the wait-and-see camp with rookies and new players. By providing the stats I see as critical to a player’s evaluation, I hope to give an impartial opinion on whether we can say we know who a player is at this point in the season. Let’s get to it!

Keaton Mitchell, RB, Baltimore Ravens

Fantasy needs another Ravens running back to worry about like we need a hole in our collective head, yet here he is: the hottest new rookie on the block. With 172 rushing yards and two rushing touchdowns on 12 carries (14.3 YPC) plus another 28 receiving yards on three targets in the last two weeks, Mitchell was the biggest waiver wire add ahead of Week 10. However, he was started in fewer than 5% of leagues that rostered him. He’s a bit undersized at 5-8 and 191 pounds — almost identical to De'Von Achane’s 5-9, 188 pounds — but more than makes up for it with speed, again echoing Achane’s claim to fame.

In the last two weeks Gus Edwards has more than double the carries of Mitchell but given that Baltimore is the league’s most run-heavy offense, there is room for two. Justice Hill will likely be the loser as a result of Mitchell’s emergence since the Ravens have been inclined to go with the hot hand(s). Mitchell’s big-play ability adds a key element to an already-potent offense.

Looking ahead, the Bengals are middle of the road versus the run, which in addition to bye-week blues is sure to make Mitchell far more startable in Week 11. Whether he ends up in the Hall of Fame is unknown, but I believe in him for now.

De’Von Achane, RB, Miami Dolphins

I know, you know, everyone knows that Achane flashed something special at the start of this season. He racked up 527 total yards and seven total touchdowns in just four games. In most of those, Raheem Mostert was nearly as good as Achane, providing a one-two punch that we don’t see too often in today’s NFL. Don’t forget that he’s potentially back this weekend to take on the Raiders, one of our favorite fantasy running back matchups.

Achane is a player I’d insert right back into my lineups — no more wait-and-see.

Kyren Williams, RB, Los Angeles Rams

Williams also returns from the IR soon (Week 12) to take away the pain of deciding between two sub-par options in the Rams’ backfield. Before injuring his ankle, Williams scored seven total touchdowns while proving valuable as a runner and a receiver out of the backfield. The dual-threat back was given a heavy workload from the jump and he handled it with increasing efficiency. He was overshadowed a little by rookie sensation Puka Nacua, whom I should note is also the real deal (fifth among WR in targets, 10th in fantasy points despite two three-reception games with Brett Rypien), but both should get back on track when Matthew Stafford returns to action.

Be sure Williams is activated and slotted into a starting position ahead of his Week 12 matchup with Arizona.

Will Levis, QB, Tennessee Titans

We’ve seen two sides of Levis in his first three games. The four-TD hero from Week 8 versus Atlanta to the four-sack zero versus Pittsburgh and Tampa Bay in Weeks 9 and 10. One of the key pieces to consider here is that the Titans didn’t believe Levis was ready to be the franchise QB and began the season behind an ineffective Ryan Tannehill. Levis was pressed into the starting role due to Tannehill’s injury, not his own improvement.

While the debut was impressive, it’s clear that there will be more growing pains ahead behind an underwhelming offensive line. With Levis unable to throw a touchdown pass in two consecutive games — while taking eight sacks — against two of the most generous pass defenses, he’s unstartable in Week 11 at Jacksonville. I’ll give him time to improve over the rest of this season and next before totally throwing in the towel because he’s actually been better overall than Tannehill. With four touchdowns and two interceptions compared to Tannehill’s two touchdowns and six interceptions, Levis is only 10 fantasy points behind Tannehill despite playing three fewer games.

CJ Stroud, QB, Houston Texans

If I’m not mistaken, Stroud has already been inducted into the Hall of Fame, right? He’s broken enough rookie records that I’m only half kidding there. Unlike Levis, the Texans could see what they had in Stroud and he hasn’t let them down since Week 1 versus Baltimore, which we now know to boast one of the best defenses in the league. Not much more need be written about Stroud — he’s an every-week starter in every league and averaging more fantasy points per game than Patrick Mahomes. Nine games are more than enough to know what Houston has in Stroud: a fantasy-league winner.

Next week I’ll be back to take a look at some older faces in new places. A small sample size of games with a new team or after a long absence due to injury can also make fantasy managers unsure of whether their player can be trusted.