It seems every pundit, news source, or so-called authority of any kind is quoting polls, surveys, or statistics these days. Anyone with knowledge of polls, surveys and statistics likely laughs themselves silly or grinds their teeth until their head hurts when one source or another speaks of the latest survey.
Survey and polling data have always been a bit suspect. Today, such data is not only suspect, much of it has the validity of all the stories claiming one city in the U. S. or another is now governed by Sharia law. Robocalling, online polls, and other modern data gathering strategies are making a mockery of these processes. In fact, after the U. S. Presidential election of 2016 many were saying polls and surveys in the political arena might be a thing of the past. As we have seen, that is not the case.
Almost every day someone is touting a new poll or survey in the media. News outlets and opinion shows constantly quote some statistic drawn from a poll or survey. One will claim a special election is too close to call. Another may claim the majority of voters disagree with this or that particular plan or proposal. Still, others opine that Republicans or Democrats, depending on the issue, feel the country is on the right track, headed for Armageddon, or on the verge of societal collapse.
Given the foregoing, one must wonder, what is going on? Why do pollsters, even those with excellent track records, find their predictions to be less reliable? Additionally, why do polls allegedly researching similar issues make predictions that seem so at odds? Polling and survey data seem to be going the way of weather forecasts which may have multiple predictions depending on the computer model used.
Several think pieces over the last few years concerning the issue of polling related to elections blame a number of factors. Most notably, the experts in this area believe the shift from traditional means of communication to mobile phones and the internet makes it more difficult to conduct polls. Not only is it more difficult to find respondents willing to answer questions, the very means of communication used can skew results.1
Without a doubt, mobile phones, social media, and online surveys are part of the problem. They are not the only problem however. Professional pollsters and researchers are running into other problems which they may or may not acknowledge. One is the question of honesty.
Traditionally, the thinking has been that a person taking part in an anonymous survey would answer questions truthfully. The belief was that people would not lie if their answer would not directly effect them. Therefore, in response to the statement, “Running a large successful business enterprise would help a person be a more effective president,” a participant would agree or disagree honestly. That is likely not the case today, if it ever was.
It is especially problematic today because people simply do not trust as they did in the past. Many believe people responding to election polling in the 2016 presidential race lied when asked about their preference for president. The thought was they were afraid to admit they were going to vote for Donald Trump because of what people might think of them. This implies they were concerned about being judged, and they did not believe their responses would be anonymous.
Anonymity may have been an issue. It is likely not the only issue. There is also reason to believe respondents will lie for less obvious reasons. Many students for example seem willing to lie about any number of issues when asked to respond to polls or surveys.
And, in the world of online communication, faulty memories or outright lies seems to be a fact of life. Why that is so may be open to debate, but research in which OneOldCop was involved some years ago indicated the perceived anonymity factor of online communication led people to act in ways they would never act in person. Still, there is a new reason to find polling and survey data problematic.
Do it yourself survey tools seem to be all the rage these days. In case you were not aware of it, a number of companies allow you, this writer, or your eccentric Aunt Edna to formulate a survey, distribute it, and then publish the results online. One can do this, without any training in data collection, understanding of research methods, or oversight. To say this is a bit concerning is an understatement.
One hopes those who poll professionally such as some nationally known organizations touting themselves as “nonpartisan think tanks” hold themselves to some professional standard. The hope would be they would not conduct surveys or polls using leading questions, or publish results designed to mislead someone about the data collected. Unfortunately, the DIY survey business allows anyone to develop a survey or poll, intentionally or unintentionally, that will result in misleading results.
That is why this writer cringes when a talking head on a morning show announces, “Coming up after the break, our latest Survey Master poll! Find out whom Americans will vote for in 2020!”
1. Interested in the expert commentary on this issue, search “the problem with polling,” or “Cliff Zukin.”
© OneOldCop – 2017
