Hillary Clinton looks like she's about to crush Donald Trump — but it might not be so simple

Before you go, we thought you'd like these...

Nearly Half Of All Registered Voters Say Trump Should Exit Presidential Race

What a difference a month makes.

Hillary Clinton, the Democratic presidential nominee, started August with a fast-rising lead in the polls. Averages of major surveys show her leading Republican nominee Donald Trump by roughly seven points. Some individual polls give her a double-digit advantage.

She wasn't always ahead, and there are cautionary tales: During the primaries, Clinton's big lead in one state's polls turned into a loss on election day.

It's because polls are more complicated than we like to think, and looking closer tells you everything about what we can and cannot trust.

Click through images of Donald Trump's speech at GOP convention RNC:

30 PHOTOS
Donald Trump delivers speech at GOP convention RNC
See Gallery
Donald Trump delivers speech at GOP convention RNC
U.S. Republican Presidential Nominee Donald Trump arrives onstage at the Republican National Convention in Cleveland, Ohio, U.S. July 21, 2016. REUTERS/Rick Wilking
U.S. Republican presidential nominee Donald Trump is greeted by Ivanka Trump after his introduction at the Republican National Convention in Cleveland, Ohio, U.S. July 21, 2016. REUTERS/Jim Young
Ivanka Trump, daughter of Republican Presidential Nominee Donald J. Trump, waves as she walks off stage after introduction her father during the final day of the Republican National Convention in Cleveland, Thursday, July 21, 2016. (AP Photo/J. Scott Applewhite)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump gives two thumbs up to the crowd during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Joe Raedle/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump acknowledges the crowd during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Joe Raedle/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump delivers a speech during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Win McNamee/Getty Images)
Donald Trump, 2016 Republican presidential nominee, speaks during the Republican National Convention (RNC) in Cleveland, Ohio, U.S., on Thursday, July 21, 2016. This evening marks the last night of a four-day Republican National Convention that has been defined by disorderly floor activity, divisions within the party, a plagiarized speech delivered by the nominee's wife and scattered protests in the streets of Cleveland. Photographer: David Paul Morris/Bloomberg via Getty Images
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump preapres to deliver his speech during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Win McNamee/Getty Images)
US Republican presidential candidate Donald Trump speaks on the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / Jim WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
Republican presidential candidate Donald Trump speaks on the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / Robyn BECK (Photo credit should read ROBYN BECK/AFP/Getty Images)
U.S. Republican Presidential Nominee Donald Trump speaks at the Republican National Convention in Cleveland, Ohio, U.S. July 21, 2016. REUTERS/Rick Wilking
The mouth of US Republican presidential candidate Donald Trump is seen on a big screen as he speaks on the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / Jim WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump delivers a speech during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by John Moore/Getty Images)
US Republican presidential candidate Donald Trump speaks on the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / JIM WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
A CODEPINK protester is removed while US Republican presidential candidate Donald Trump speaks on the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / JIM WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump delivers a speech during the evening session on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Alex Wong/Getty Images)
Delegates listen as Republican presidential candidate Donald Trump addresses the final night of the Republican National Convention at Quicken Loans Arena in Cleveland, Ohio, July 21, 2016. / AFP / Robyn BECK (Photo credit should read ROBYN BECK/AFP/Getty Images)
Republican U.S. presidential nominee Donald Trump formally accepts the nomination at the Republican National Convention in Cleveland, Ohio, U.S. July 21, 2016. REUTERS/Carlo Allegri
Republican U.S. presidential nominee Donald Trump formally accepts the nomination at the Republican National Convention in Cleveland, Ohio, U.S. July 21, 2016. REUTERS/Carlo Allegri
CLEVELAND, OH - JULY 21: Attendees listen to Republican presidential candidate Donald Trump deliver his speech on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Jeff Swensen/Getty Images)
CLEVELAND, OH - JULY 21: Deligates stand and cheer as Republican presidential candidate Donald Trump delivers his speech on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Win McNamee/Getty Images)
CLEVELAND, OH - JULY 21: An attendee stands and cheers as Republican presidential candidate Donald Trump delivers his speech on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Chip Somodevilla/Getty Images)
Rudy Giuliani, right, former mayor of New York City, applauds as Republican Presidential Nominee Donald Trump addressed the audience during the final day of the 2016 Republican National Convention at Quicken Loans Arena in Cleveland, Ohio, July 21, 2016. (Photo by Brian van der Brug/Los Angeles Times via Getty Images)
CLEVELAND, OH - JULY 21: (L-R) Tiffany Trump, Barron Trump and Melania Trump listen to Republican presidential candidate Donald Trump deliver his speech on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Jeff Swensen/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump and his family acknowledge the crowd on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Jeff Swensen/Getty Images)
US Republican presidential candidate Donald Trump (C L) and vice presidential candidate Mike Pence (C R) are joined by their families at the end of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / JIM WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
Republican presidential candidate Donald Trump embraces his wife Melania on the final night of the Republican National Convention at the Quicken Loans Arena in Cleveland, Ohio on July 21, 2016. / AFP / Jim WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump and Republican vice presidential candidate Mike Pence stand with their families on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Chip Somodevilla/Getty Images)
Balloons descend on the delegates as the families of Republican presidential candidate Donald Trump and his nominee for vice-president Mike Pence appear on stage at the end of the last day of the Republican National Convention on July 21, 2016, in Cleveland, Ohio. / AFP / DOMINICK REUTER (Photo credit should read DOMINICK REUTER/AFP/Getty Images)
CLEVELAND, OH - JULY 21: Republican presidential candidate Donald Trump embraces Republican vice presidential candidate Mike Pence after his speech on the fourth day of the Republican National Convention on July 21, 2016 at the Quicken Loans Arena in Cleveland, Ohio. Republican presidential candidate Donald Trump received the number of votes needed to secure the party's nomination. An estimated 50,000 people are expected in Cleveland, including hundreds of protesters and members of the media. The four-day Republican National Convention kicked off on July 18. (Photo by Jeff J Mitchell/Getty Images)
of
SEE ALL
BACK TO SLIDE
SHOW CAPTION +
HIDE CAPTION

Here's a poll from before Clinton's primary bounce: Rasmussen Reports asked 1,000 Americans in mid-July whether they would vote for Clinton or Trump. This is what it found at the time:

  • Trump: 43%
  • Clinton: 42%

Was Trump ahead then? Is Clinton ahead now?

It depends on whom you ask

In 1936, a magazine called The Literary Digest ran one of the biggest opinion polls of all time. It asked 2.4 million people whether they planned to vote for the incumbent Democratic president, Franklin D. Roosevelt, or his Republican challenger, Alfred Landon.

BI Graphics Lit Dig coversDragan Radovanovic / Business Insider

It trumpeted this prediction:

  • Landon: 57%
  • Roosevelt: 43%

The poll must have had one of the smallest margins of error in polling. But it was dead wrong.

Error margins apply only to the population a pollster is sampling.

This is what actually happened in the election:

  • Roosevelt: 62%
  • Landon: 38%

The Literary Digest fell prey to what is known as selection bias. That massive sample was made up of its subscribers and members of groups and organizations that tended to skew wealthier than the average American.

Today's pollsters are savvier, but there are still many ways that bias seeps in. For instance, a poll that calls only landlines may leave out a whole demographic of younger, cellphone-only households. Some polls are opt-in, where users of a specific website answer questions. That's less reliable than a random sampling.

"Far more important than dialing down the margin of error is making sure that whatever you're aiming at is unbiased and that you do have a representative sample," says Andrew Bray, an assistant professor of statistics at Reed College.

Some polls have well-known biases. Rasmussen, for instance, is known to skew Republican.

Lee Miringoff, the director of the Marist Institute for Public Opinion — which produces polls for NBC News, The Wall Street Journal, and McClatchy — says polls are as much art as science.

"Scientifically, we should get the same result," he says.

Modern polls are not immune to these issues. Some potential voters are harder to reach, and some polls skew more educated. And polls with a high percentage of potential voters who are undecided can lead to more uncertainty.

So how much can we trust today's results?

Margin of error

BI Graphics_Spearfishing vs throwing a net_transparentDragan Radovanovic / Business Insider

Pollsters and journalists tend to highlight the headline numbers in a poll. If you did that with the July Rasmussen example, you'd think Trump was leading Clinton.

Rasmussen didn't help matters by describing Trump as "statistically ahead."

It's actually not that simple.

First, you have to consider the margin of error. Rasmussen pollsters interviewed 1,000 people to represent the views of 320 million Americans. Naturally, the poll results might not perfectly match what the whole population thinks.

This Rasmussen poll has a 3-point margin of error. Here's what that actually means.

Let's take that Trump number: 43% is something called a "point estimate." This is basically the polling firm's best educated guess of what the number would be if it had asked the whole population. But it's not guaranteed to be right.

The margin of error accounts for this:

  • Because the margin of error is 3 points, the pollsters are confident that support for Trump in the total population is between 40% and 46% — or 43% plus or minus 3 percentage points.
  • Support for Clinton is between 39% and 45%.

confidence intervalsDragan Radovanovic / Business Insider

The point estimate (the dots in the chart above) is like fishing with a spear; you're stabbing for the right answer. The margin of error is like fishing with a net; somewhere in your catch is the true figure.

But this is not the whole story either.

Feeling confident

Before the 2016 Michigan primary, it looked as if Clinton had it made. FiveThirtyEight aggregated several polls and predicted that she had a 99% chance of winning the primary. Many polls had Clinton ahead of challenger Bernie Sanders by double digits.

The polls were wrong.

Sanders eked out a narrow victory. Of the many reasons pollsters might have been off, this may be one of them: There's more to polling than the margin of error.

"The margin of error is a guidepost, but not a foolproof" one, Miringoff says.

Here's what the margin of error really means.

Pollsters typically ask roughly 1,000 people a question like: Whom do you plan to vote for? Their goal is to be 95% sure that the real level of support in the whole population is captured in the sample's range, from the low end of the margin of error to the high end.

That range is called a "confidence interval."

Let's say a pollster like Miringoff were to run that same poll 100 times. Each time, he would randomly select different groups of 1,000 people. Miringoff would expect that the true proportion — the candidate's actual support — would be found within the margin of error of 95 out of the 100 polls. That's why he'd say that he's 95% confident in the results.

Those five outliers are one reason elections don't always turn out the way pollsters predict.

Remember that Rasmussen poll in July showing Trump with 43% support? That 43% is thought to be the most likely reflection of reality. But the pollster is still only 95% confident that Trump's true amount of support is found between 40% and 46%.

The further you get from that point estimate, the less likely it is that you are seeing the true number. So it's more likely to be 42% than 41% — and 40% is even less likely.

normal curve annotated 2Dragan Radovanovic / Business Insider

The chance that what's happening in reality is captured by a number outside the 95% confidence interval is, as you might expect, quite unlikely. The more outside it is, the more minuscule the likelihood. But it's still possible for a poll to be way off.

"If you really want to be 100% confident in your estimate, you're either going to have to ask every American or be satisfied with a huge margin of error," Bray, the Reed College statistics professor, says.

The whole point of polling is to extrapolate what a large group believes by asking a randomly selected subset of that group.

In the era of modern polling, most pollsters agree that being 95% confident in the margin of error is "good enough."

"It's a reasonably high number," Bray says. "That means we're going to be wrong one in 20 times, but for most people that's acceptable."

Many polls, such as those from the Pew Research Center, bury the margin of error in the fine print. Far fewer highlight the confidence interval. But anytime you see a poll, remember: There's a 5% chance that the poll is far different from the headline number.

Keeping it 1,000

BI Graphics_1000 emoji_transparentDragan Radovanovic / Business Insider

Look closely and you'll notice that most polls question roughly 1,000 people. That holds true whether pollsters are trying to approximate voter opinion in Rhode Island (about 1 million residents) or the entire US (nearly 320 million residents).

Why 1,000?

It's a big enough number to be reasonably confident in the result — within the margin of error 19 out of 20 times. There's a lot of variety in a group of 1,000 people, so it captures many of the elements in the larger group.

Asking more people than 1,000 leads to diminishing returns of accuracy.

For instance, sampling 2,000 people is not twice as precise as sampling 1,000. It might bring the margin of error from roughly 3 points to about 2.2 points.

moe vs sample size 3Dragan Radovanovic / Business Insider

In modern polling, most statisticians see sampling 1,000 people as a good compromise between a manageable sample size and acceptable confidence.

What and when

Results differ among pollsters for many reasons.

There are simple explanations, like when the polls were conducted. It can take days or weeks to conduct and analyze a poll. A lot of news can happen between the dates on which the questions were asked and the date of the results' release.

BI Graphics_Polling Days vs Election Day transparentDragan Radovanovic / Business Insider

This is especially a problem with polls close to Election Day. They're generally a snapshot in the week before the election. If something happens in the final days of campaigning, those final polls may not be as predictive.

It also matters how a pollster phrases and orders questions, and whether it's a phone interview, in-person interview, or online survey. Even the interviewer's tone of voice can matter.

Then, pollsters have to decide how to analyze and weight the data, and those methodologies can vary.

But it's not just pollsters analyzing data — and that's where we get another big problem.

Drilling down

BI Graphics 1000 GroupDragan Radovanovic / Business InsiderWhen Miringoff releases his Marist polls into the wild, they are quickly consumed by journalists, commentators, and a public looking for trends that create headlines. This drives him crazy.

"It's too often to throw up your arms," Miringoff says.

Here's the problem: Let's say his team interviews 1,000 people to represent the general population. In that 1,000, there are subgroups: men versus women, minorities, immigrants, young people, old people.

It's tempting to pull out those subgroups and draw conclusions about, say, support for a candidate among Latinos or women.

But each of those subgroups is, in effect, its own sample, and those samples can be very small. That means the margin of error for each subset can be huge.

Take this poll from Pew: In the sample, there were only 146 black respondents. The margin of error for that subgroup is more than 9 points!

redo screenshotDragan Radovanovic / Business Insider

You can't learn much by looking at a group with a 9-point error margin.

Why aggregating is good

If you combine results from multiple polls taken at the same time, you can think of it as one huge poll. That drives down the overall margin of error and can make you more confident in the predictive power of the polls.

In the real world, different polls are conducted in different ways, so you can't think of an aggregated poll as truly one big sample. But this is also a virtue because it reduces the effect of pollster biases and errors. FiveThirtyEight, The New York Times, and RealClearPolitics all run averages with different weightings and methodologies.

Ahead or tied?

OK, so now that you know a lot more about polls, what should you think when a race is tight? The answer is not straightforward.

Let's say that a poll comes out showing Clinton with 51% support and Trump with 49%. The margin of error is plus or minus 3 points. Are the two candidates statistically tied, or is Clinton slightly ahead?

In purely statistical terms, most would consider this example a "statistical dead heat." Either candidate could be ahead.

"It's pretty significant editorially," Miringoff says. "It's not significant statistically."

That said, that doesn't mean Clinton's lead in this hypothetical example is completely insignificant.

"If I was running for office, I'd rather have 51 than 49," Miringoff says.

Remember point estimates? In this scenario, 51% is still the pollster's best guess at Clinton's true level of support. That's higher than Trump's. If a series of polls shows Clinton with a slight edge — even within the margin of error — then it can suggest an advantage. A series of polls is more convincing than any single poll.

Feedback loop

BI Graphics_Trump Feedback Loop transparentDragan Radovanovic / Business Insider

Finally, for as much as we want to believe that polls are a scientific reflection of reality, polls can also affect reality.

Here's one example: Polls that show candidates falling behind can galvanize their supporters to get out to vote.

The media may also focus on polling trends, leading to changes in public opinion about which candidates are viable or worth supporting.

Polls don't happen in a vacuum.

NOW WATCH: INSTANT POLL: Americans viewed Clinton's convention speech more favorably than Trump's

See Also:


Read Full Story

People are Reading