If you’re lucky, you’ll only receive a few spam calls this week. Billions of robocalls are made to cell phones and landlines every month. Many of these are spoofed to look like they’re coming from the target recipient’s area. So it’s no surprise we have collective trust issues preventing us from answering our phone when a stranger calls to ask questions about our political views.
Political polling happens all year round, but in the months preceding elections, polls become something of an obsession. In the weeks leading up to elections, Americans may follow poll fluctuations day-to-day, and see the stats on the daily news. But how is the field of public polling faring in a time when public attitude towards talking on the phone is shifting? It’s a problem that’s likely only going to get worse—one recent study projects that almost half of the calls made to mobile devices in 2019 will be scams. And it’s not like Americans have much faith in the survey research field after they were led to believe that polls showed Donald Trump had very little chance of winning the 2016 presidential election.
Last year, Pew Research Center released a study about response rates to telephone polls. The report showed that rates have plummeted since 1997, as United States citizens have ditched landlines for mobile phones en masse, and generally grown less willing to talk on the phone.
But the report showed that after response rates had steeply dipped from 36 percent in 1997 to 6 percent it 2012, the rate had stabilized through 2016. Politico wrote about the report, claiming it showed that“phone polls aren’t dead yet.”
The article connects our growing collective distrust for unrecognizable numbers to the telemarketing craze that began 20 years ago. But what was once a craze is now a zombie plague, as new software has made it virtually free for telemarketers and scammers to spoof numbers and send out millions of calls, many of which seem to come from local numbers. The robocall issue has gotten much worse since Pew collected data about response rates, and one of the authors of the Pew report—Pew research director of survey research Courtney Kennedy—told Gizmodo that the latest polls show that response rates have continued their decline since the report came out. And she said the decline is partly attributable to robocalls.
“A lot of these sort of technological changes into the telephone environment have really taken effect in 2017 and 2018,” Kennedy told Gizmodo. “And so when we updated our latest polls in 2017 and 2018, the response rate has resumed its decline.”
Kennedy points to two factors that have likely played a part in the decline of response rates. The documented increase in robocalls, and cell phone carriers’ efforts to alert customers’ of numbers that could be spam calls, a functionality that can also flag calls from polling firms.
So are polling firms concerned about the plague of robocalls?
“Very concerned. It’s reducing response rates and it is increasing costs in a noticeable way,” Kennedy said. “We were challenged before then. But it’s clear that those problems have increased.”
Of course, when interviewers have to make more calls in order to get enough responses, labor costs go up. Polling firms call more cellphones than landlines now that most Americans have cellphones. But according to Pew, a cellphone interview can cost twice as much as a landline interview, because of federal regulations that require research firms to dial cell numbers by hand, while they can still use auto-dialers to call landlines before a human interviewer takes over.
Kennedy and other polling experts said the costliness of phone polling is driving some firms to embrace cheaper methods—namely, online polls and automated calling, also known as interactive voice response (IVR). When Gizmodo started talking to polling researchers, we were curious about the influence that robocalls were having on the polling industry. However, while growing societal reluctance to answer phones is making it more difficult for polling interviewers to get a sufficient number of responses, experts seem more concerned with growing popularity of online polls—like SurveyMonkey and Google Consumer Surveys polls.
You’ve likely seen one of these types of polls. You were trying to access a news article or you were invited to answer questions while shopping. Sometimes the queries are about brand recognition or technology, but they can also be about political beliefs and voting plans. These types of polls lack the scientific sampling of random digit dial telephone surveys. They’ve also proven to be highly inaccurate when used for state polls, according to FiveThirtyEight, which said in May it was contemplating whether or not to include those types of polls in forecasts.
During phone surveys, an interviewer aims to stoically ask a series of questions. In Pew calls, the interviewer may start out by asking about the respondent’s approval of the president, then ask about other topics, like the economy or immigration. The interviewer generally finishes the call with demographic questions that help with weighting the data (like age, education, and race) and demographic questions that could be used for analysis (like political party affiliation, and religious beliefs, and labor union membership).
An operation that does such thorough person-to-person research, on a large scale needs robust resources. But new technology is providing a cheaper alternative, and many career pollsters think we could be sacrificing a lot with the transition to these new types of polling techniques.
“When I got into the business about 20 years ago, you had to have a brick and mortar-type business. You had to have calling stations, you had to have years of experience in the field. You really had to know what you were doing,” Kennedy said. “And those things are all gone because of technology. Now you see pollsters pop up seemingly overnight and then put out a press release with poll numbers. Because anyone can buy a poll these days, using online opt-in sample, or using automated polls.”
David Dutwin, chief methodologist at SSRS research, has a nickname for the facilitators of these types of polls. “I call them the heroin dealers—who are providing very low-cost, non-scientific survey data, and convincing media organizations that these are worthy of coverage, increasingly convincing academics that they should rely on these data.”
“I am angry at the internet right now,” Dutwin said. “It’s visibly having an impact that is undermining democracy to some extent.”
Dutwin acknowledged there are “a lot of different versions of that story” of the internet killing democracy, but his version is related to survey research. He references politicians’ responses to the Parkland shooting earlier this year: “I’d rather live in a world where if there’s a mass shooting in Florida and legislators in Washington [DC] are thinking seriously about whether to pass gun control legislation or not—I would like for there be an opportunity for those legislators to quickly learn from valid scientific poll that gives every American an equal voice and express their opinion on what they would like to see done on that issue.”
Dutwin says the random digit dial telephone survey done by human interviewers is still the gold standard for survey research. Leading up to the 2016 presidential election, that method returned results that had an average error of slightly less than one percent—about the same as what pollsters saw in the 2012 and 2008 presidential elections. However, he found that 2016 pre-election polls that were done through internet sampling had an average error of 5 percentage points.
“So that method, where there is no scientific sampling, is what caused the problem in 2016,” Dutwin said. “They’re the overwhelming number of polls publicized in 2016 that caused everybody to go to bed the night before thinking the polling aggregators say there’s at least a 70 percent, 90 percent chance that Hillary Clinton is going to be president. They were wrong because they were all based on these non-scientific polls.”
While national news and polling organizations still have resources to do random digit dial telephone surveys—local news organizations are increasingly turning to online surveys.
“National pollsters are significantly more likely to do live polling than polls done at the state level,” Pew’s Kennedy said. “A typical state poll is, like, the local newspaper in Cleveland and they basically don’t have anywhere near the resources to put into their polling. And that translates into more automated, in some ways less rigorous methods being used and that has implications that’s one of the big stories of the 2016 election.”
But it’s not just our ability to predict elections that is suffering. Policy makers are losing a sense of what U.S. citizens actually want. “Ultimately if things don’t get better, the public is going to be hurt by this,” said Michael Traugott, president of the World Association for Public Opinion Research and professor at the University of Michigan. “If you go back to the development of the polling industry in the United States 80 or so years ago—the belief was that quality polls would serve the public by allowing them to express their views that government leaders in between elections.”
When polling integrity decays, it becomes easier for powerful people to ignore the public, according to Jon Krosnick, director of Stanford University’s Political Psychology Research Group. “A modern democracy is really based on knowing what people think,” Krosnick said. “And knowing what people think is important because then, elites can’t just say whatever they want—say, ‘Oh you know the people think X.’ No, we have polls that tell us what people think.”
Unsurprisingly, pollsters have a very idealistic view of the importance of political polling to a healthy democracy. Even if we have accurate representative polling—it’s possible the majority of policymakers still won’t care what the constituents really want. For instance, Gallup polls have consistently shown since the 1990s that the population of Americans supporting stricter gun control laws far out number those supporting looser gun control laws. And yet, politicians have largely ignored the desire of the American public.