Is that big lead for Sharice Davids over Amanda Adkins just a polling mirage?

Did you see the recent New York Times poll that gave Sharice Davids a whopping 14-point lead over Amanda Adkins in the Kansas 3rd Congressional District race?

The number was astounding, and maybe a bit unbelievable.

Davids beat Adkins by 10 points in 2020. But after that, the Kansas Legislature redrew the district to give Republicans a better shot at winning the seat this year. And Democrats nationally are bracing for a tough election that could likely end their majorities in Congress. Despite all that, The Times’ poll suggested that Davids might somehow outperform her results from two years ago.

There’s reason to think the poll might not be that accurate, however.

Why? Because the poll has all the hallmarks of “nonresponse bias,” The Times’ Nate Cohn explained in a newsletter published Tuesday. Basically, Democratic voters in the 3rd District were far more likely to answer pollsters’ questions than Republicans, and that gap could well have skewed the final numbers toward Davids.

Nonresponse bias is a national issue for pollsters — it probably explains why 2020 polls memorably underestimated Donald Trump’s support in the presidential election — but the problem really showed up in the recent Kansas poll: Democrats were 70% more likely than Republicans to answer questions.

That’s a huge number.

“I’m pretty sure that’s the largest disparity in partisan nonresponse we’ve ever encountered,” Cohn wrote, adding that Davids’ apparent double-digit lead was “a solid 10 percentage points more than I would have guessed before we fielded the poll.”

Polls — and creating and taking them is a real science — are supposed to eliminate guesswork. Done properly, they should accurately predict, within a margin of error, what voters are thinking at a given time. The best journalism avoids polls, no matter how sexy the findings are, until the data scientists creating the polls have done their job properly.

In this case, we’re thus left with two problems — one for journalism, and one for democracy in Kansas.

Let’s start with the democracy issue. Polls aren’t just snapshots of public opinion, as pollsters often like to say. They also influence that opinion. When voters see big gaps between candidates in reported polls, they can “switch sides in an effort to feel accepted and to be part of a winning team,” Stanford researchers found in 2012.

If The Times’ poll is wrong, then, Davids might get an artificial boost from bandwagon hoppers. That hardly seems fair to Adkins.

Another possibility cuts the opposite direction, though. Midterm elections are often characterized by low voter turnout, which makes it all the more critical for candidates and their campaigns to get their supporters to the polls. Enthusiasm and motivation matter. It can be harder to motivate your voters to show up if they think you’re on the way to a blowout win. If an election is actually closer than a poll indicates, that lost enthusiasm might matter a lot.

“Especially in a newly-drawn district, we aren’t taking anything for granted,” Davids’ spokesperson told The Kansas City Star last week.

Even with those stakes, it’s probably unreasonable to expect the media to shy away from reporting poll results during elections. But if there are caveats about the results — like a 70% disparity between Democratic and Republican response rates — it’s probably best to declare them up front and let readers decide how best to weigh the information.

The Times got that part only half-right. The paper acknowledged qualms about the poll in Cohn’s newsletter, but didn’t mention them in its initial news story on the results.

Perhaps The Times’ numbers will turn out to be reasonably accurate. Pollsters do try to correct for partisan imbalances in their surveys, a big challenge given not just Republican reluctance to participate but also the rise of technologies like cell phones and caller ID that make it easy for voters to avoid unwanted inquiries. It seems wise to take the results with a grain of salt, and to remember: Never let a poll do your thinking for you.