The Power and Pain of Polling
In North America we poll on almost anything we can imagine. What do we think is the softest toilet paper? What’s our favourite soft drink? How do we like our spaghetti? Who do we think we’ll vote for in the next election? We would all like to believe that polling results are either completely useless or absolutely insightful. Sometimes they are both. After all, an insight into how I feel about something today is not necessarily an indicator of how I will behave tomorrow. So we poll again tomorrow to see if the answer has changed and if it has, we poll again in an attempt to determine what influenced that change.
The concept of public opinion as we understand it was first coined by French Foreign Affairs Secretary, Jean-Jacques Rousseau in 1774 and American newspapers started using polling techniques in the 1820s. However, polling in Canada didn’t really take root until the 1960’s. In 1959 the Quebec Liberal Party first used a poll to determine their election strategy for the 1960 election; by 1965 Canadian newspapers were well engaged in the practice of polling. Today, newspapers and political parties are strong adherents to the power of polling and most polling companies have an affiliation with either a newspaper or a political party. There is of course polling related to commercial ventures and while these are lucrative activities and therefore dominate the polling landscape, political polling remains an active portion of all polls conducted.
Despite the popularity of polling there are always questions around the accuracy of polls. Methodology is often called into question if companies conducting similar polls get significantly different results. These differences are a reflection of the quality, tone and tenure of the questions asked. The answers are also influenced by who was asked. After all, if you were to poll a riding that has strong conservative roots, you are unlikely to obtain results that support Liberals and visa versa.
In May 2013, folks in British Columbia were surprised to discover that they still had a Liberal government after the votes were counted in their provincial election. Their surprise and that of the pundits and pollsters came because polls had been clear that the New Democratic party was well in the lead.
“I haven’t trusted polls since I read that 62% of women had affairs during their lunch hour. I’ve never met a woman in my life who would give up lunch for sex.”
Erma Bombeck, U.S. Humourist
One of the most popular approaches or survey techniques is to randomly select individuals and ask them a series of carefully crafted questions. The questions are generally created to avoid bias in the answer. The responses are then tabulated and reported on. While there are a number of ways of gathering information, the thing to consider when you look at polls is that no amount of data, regardless of how accurately gathered will be worthwhile unless you can interpret results accurately. In effect, if the act of polling is a science, then the analysis of polls is an art. The ability to not only interpret but achieve insight into how respondents are likely to react is what makes good polling companies worth their weight in gold.
If you are unclear about what I mean, consider the difference between two questions:
- Do most Canadians want their elected officials to behave in a decorous fashion?
- Do most Canadians expect their elected officials to behave in a decorous fashion?
Although the questions appear to be almost identical, they will likely elicit completely different responses. What Canadian’s want and what they expect to get can be completely different things. Do you want your kids to listen to everything you say? Do you expect them to listen to everything you say? Understanding these distinctions is what real pollsters bring to the table.
Even when conducting an apparently random poll, pollsters must consider who is actually answering questions. For the most part, many of the polls conducted today are based on random digit dialing, this means you are unlikely to participate in a poll unless you use a land-line. You may correctly assume that people who use a land-line include the majority of citizens in North America, but the question is, are those who rely solely on mobile phones a homogenous group, do they represent a distinct portion of the population? As it happens, they do. The dominant characteristics of those who use a mobile phone almost exclusively, include being young, single and not a home owner. This group is estimated to make up about 7% to 9% of the US population. While this may seem insignificant, remember that these folks are the same people who don’t vote and cannot relate to the issues that most politicians see as priorities. What happens when a generation of people who don’t vote and whose opinions we can’t predict through traditional channels grow up?
What do you think about polling? Do you pay attention to polls? Do you participate in them?
- BC pollsters revising methods after unexpected Liberal majority (globalnews.ca)
- Gallup Tries To Explain Why Its Polling Was So Bad In 2012 (outsidethebeltway.com)
- Gallup’s 2012 election polling errors were only part of the problem | Henry Enten (guardian.co.uk)