Skip to content

It’s the political scientist in me

Richard Fulton

You may have noticed that we are bombarded with polls these days. I tend to ignore statewide polls on state officials this early in the game since most people don’t pay much attention to these races until after Labor Day.

National polls, however, blossom after the party conventions. The early polls after these events tend to give a “bump” to the candidates. But then things normalize, or get as normal as polls can get.

Political wisdom has held that the candidate ahead a couple weeks after the conventions, when things settle down, tends to be the winner come November. This, of course, seems to be anything but a normal election year.

It may be instructive, then, to talk a bit about polls themselves. Pollsters have had a difficult time in recent years because most have depended upon techniques that don’t quite work anymore and therefore they must adapt.

For instance, most national polls have been conducted over the landline phone—the kind most everyone has had in our homes for the last seventy years or so. But many have dumped those phones completely for hand-held mobile phones.

These phone holders, younger and more mobile than home based users, are harder to reach and harder to get to take the survey. Historically, the more accurate polls have been in-person interviews. But fewer people want to talk to a stranger who comes knocking at their door. And they are more expensive to do.

What happens, then, is that most polls are taken by phone, but because of the problems of getting the right demographics, getting people on the go to take the survey, and a more general suspicion of pollsters, the number of samples taken must be increased to get more confident results. Increasing the cost of the surveys.

Remember, polls operate by a formula that tries to match the characteristics of the survey respondents with the characteristics of the target audience. So if you want to survey “probable voters” or “registered voters” or “qualified voters” you need to find the characteristics of the whole of this body of people.

Each of these categories have a different number and a different set of characteristics than the other categories. “Registered voters” is a larger group, for example, than “probable voters.” And they tend to be different in overall characteristics. Pollsters know that a great many registered voters do not, in the end vote. And even more “qualified voters” do not register or vote.

There are differing age, income, and ethnic differences as well.

So when you read a poll you need to know which group is being surveyed. And you need to have some idea of how much error is in the total process of surveying this group. Most professionals aim for a 3% error factor (if the error factor is not reported, don’t read or listen to it). This means the reported numbers could be 3 percentage points higher OR lower than the reported number and still be accurate. A six point range.

If a poll reports that Clinton leads Trump by three points (43-40) then statistically the results of the survey could (and still valid) be Clinton 40% to Trump 37%; or it could be Trump 43% to Clinton 40%. The range for Clinton is 37%-46%, for Trump 37%-43%. The pollsters are saying that the more likely accurate numbers are Clinton 43% to Trump 40%.

The larger the error factor, then, the less reliable is the survey results. At 4% the survey is not too telling. More than 4% the survey is basically useless.

A more hidden and difficult to measure error factor is the composition of the survey itself. Are there leading or “hot button” words or phrases that try to point the respondent to particular responses? Happens all the time.

Survey research is an art, not a science. Each survey organization has its own formula for putting together the survey and finding the sample of respondents to answer the survey questions. Thus different results reported.

And these are the more legitimate pollsters. More on this next week.

Leave a Comment