(Mis)reading polls

A reporter recently sent me a long list of detailed questions about how to tell if a political poll is reliable.

Things like margin of error, sample size, live vs. robocalls, question wording, question order, the pollsters’ track record, partisan affiliation, past performance, etc.

All good stuff. But face it, you’re not going to that much trouble.

So how do you know which polls to trust? Here’s my advice.

Don’t pay too much attention to individual polls. Or to small changes in the numbers, which the media too often treats as real changes in the race.

To wit: “Clinton was 7 points ahead in poll last week, and today she’s only 5 points ahead, so Trump is gaining!”

Not necessarily. That’s meaningless. All too often it creates a false narrative. Reporters and editors, after all, don’t want to keep writing the same story. They want news. There’s a reason it’s called “news.”

Look at big trends. And averages.

For example, Clinton has had a significant lead (in this polarized political environment, anything more than 2-3 points is significant) since the conventions. That hasn’t changed in a month.

Another example, where there has been a big change. Before Governor McCrory signed HB2, he was ahead in most every poll. Since then, he has been behind Roy Cooper in every poll. Sometimes by as much as 7 points.

It’s also big that Deborah Ross has gone from being behind in every poll to within a couple of points, up or down, with Richard Burr.

Watching the polls every day, or hour, is like watching the stock market every day, or hour. It’ll make you crazy.

Take them all in, but don’t take one or two blips up or down too seriously. Check in about once a week, say on Tuesdays, to get the big picture.

If you’re hopelessly addicted to the minutiae of polling, take a deep dive into the website FiveThirtyEight, especially Nate Silver’s analyses.

 

Posted in
Avatar photo

Gary Pearce

Categories

Archives

Recent Posts

(Mis)reading polls

A reporter recently sent me a long list of detailed questions about how to tell if a political poll is reliable.

Things like margin of error, sample size, live vs. robocalls, question wording, question order, the pollsters’ track record, partisan affiliation, past performance, etc.

All good stuff. But face it, you’re not going to that much trouble.

So how do you know which polls to trust? Here’s my advice.

Don’t pay too much attention to individual polls. Or to small changes in the numbers, which the media too often treats as real changes in the race.

To wit: “Clinton was 7 points ahead in poll last week, and today she’s only 5 points ahead, so Trump is gaining!”

Not necessarily. That’s meaningless. All too often it creates a false narrative. Reporters and editors, after all, don’t want to keep writing the same story. They want news. There’s a reason it’s called “news.”

Look at big trends. And averages.

For example, Clinton has had a significant lead (in this polarized political environment, anything more than 2-3 points is significant) since the conventions. That hasn’t changed in a month.

Another example, where there has been a big change. Before Governor McCrory signed HB2, he was ahead in most every poll. Since then, he has been behind Roy Cooper in every poll. Sometimes by as much as 7 points.

It’s also big that Deborah Ross has gone from being behind in every poll to within a couple of points, up or down, with Richard Burr.

Watching the polls every day, or hour, is like watching the stock market every day, or hour. It’ll make you crazy.

Take them all in, but don’t take one or two blips up or down too seriously. Check in about once a week, say on Tuesdays, to get the big picture.

If you’re hopelessly addicted to the minutiae of polling, take a deep dive into the website FiveThirtyEight, especially Nate Silver’s analyses.

 

Posted in
Avatar photo

Gary Pearce

Categories

Archives