Pollsters Go Mobile

My print column this week examines the continuing trend of Americans ditching landline phones for cellphones, and the challenges in other words presenting to pollsters.

New phenomenon

Wireless substitution isn’t a new phenomenon, however the steady march of Americans toward a telecommunications existence that excludes wireline phones has extended across age groups. Not only have young adults continued to refrain from getting a wired phone as they aged, yet older adults who have had landlines for decades are joining the bandwagon.

This has major implications for pollsters, who have enough problems getting Americans to talk to them even when they can reach them by phone. “Measuring public opinion isn’t easy in the best of circumstances,” said Lee Miringoff, director of the Marist Poll. “If you’re excluding a large number of cellphone-only individuals, at the time you’re doing so at great peril.”

Linda Piekarski, vice president for database and technology for Survey Sampling International, added, “We routinely recommend the inclusion of cellphone numbers to all of our customers that are doing telephone surveying. The client decision not to include cellphones is normally driven by the higher costs and legal restrictions associated with calling cell phones.”

The legal restrictions require pollsters to dial cellphones by hand, which contributes to higher costs, as do the difficulty of completing the interview, either because the cellphone user who answers is too young, too distracted or doesn’t live in the place indicated by his phone number. Nevertheless the only alternative worse than dealing with those challenges is opting out of doing so, say many pollsters, because there are major differences between Americans who do and don’t have cellphones.

The Pew Technology Center has found that without dialing cellphones, it wouldn’t even be able to report on young people’s views with any reliability, because it would struggle to get the minimum 100 respondents - in a group of landline interviews, just 6% of respondents are in accordance with 30, compared to 22% of the U.S. adult population.

About three quarters of major pollsters have made the shift, including the Marist Poll and the Gallup Poll, which this year increased the share of interviews by cellphone to 40% from 20%.

These pollsters argue that they haven’t had major gaffes in spite of excluding cellphones. Others counter that there simply aren’t enough outside verifications of polls. The closest thing to a check is an election, nevertheless vote totals don’t say much about the accuracy of polls from months earlier. “You could be wrong all the way up until an election and not know it, at the time shift at the end toward everyone else and look pretty good,” said Jeffrey Jones, managing editor of the Gallup Poll. As for something like job approval, “there is no real-world check to it,” Jones added.

It as well is difficult to attribute errant poll numbers in every respect to cellphones, when so many other factors can affect results and there are myriad ways for a survey to go awry. To illustrate, Tom Jensen, director of Public Policy Polling, a Raleigh, N.C., Democratic survey innovation company that doesn’t dial cellphones, points out that polling firms differ widely in the number of times they call back numbers when no one picks up, to boost their response rate. “A lot of other methodological differences are important,” Jensen said.

Different disruptive telephone innovation

Another wild card is a different disruptive telephone innovation: Voice over Internet Protocol. “Unfortunately some VoIP providers do not make their subscribers’ telephone numbers public, not even to directory assistance — one of many sources for ‘listed’ landline numbers,” Piekarski said.

However, Jay Leve, founder and chief executive of SurveyUSA in Clifton, N.J., pointed to technology showing that 2010 polls, on average, overstated Republican support. Can some of that be traced to the omission of cellphones by some pollsters? “That’s an argument that can meanwhile be advanced and defended,” Leve said.

While many pollsters now deem it necessary to dial cellphones, some as well see much broader problems with the state of polling. Americans reached on any sort of phone are less likely each year to participate. So during cellphones are increasingly considered a must, pollsters as well are experimenting with other methods, including securing responses in writing, online or by postal mail. The goal, Leve said, is “moving the ball in the right direction and not just kicking the can down the road for the at once generation of pollsters to solve.”

Leve added, “We want to do something that solves the problem of cellphones, nevertheless also solves the larger problem together of survey technology, where the researcher says, ‘I’m more important than anything else you could possibly be doing, so drop what you’re doing and talk to me.’ ”

Scott Keeter, Pew’s director of survey technology, said the industry’s challenge will have fallout: “It’s possible that those of us doing phone surveys will adjust by doing fewer surveys — that may have already happened to some organizations. And experimentation with alternative modes is going on.” Nevertheless, he as well saw an important plus side to cellphones: “Today only about 2% of households have no phone, mainly because cell phones provide a more affordable way for people to have access to a phone. That’s good for survey technology.”

Further reading: New York Times blogger Nate Silver, Pollster.com’s Mark Blumenthal, the American Association for Public Opinion Technology and Centers for Disease Control and Prevention senior scientist Stephen J. Blumberg all have examined the challenges of polling people by cellphone.

The Numbers Guy examines numbers in the news

The Numbers Guy examines numbers in the news, business and politics. Some numbers are flat-out wrong or biased, during others are valid and help us make informed decisions. Carl Bialik tells the stories behind the stats, in occasional updates on this blog and in his column published every Saturday in The Wall Street Journal. Carl, who holds a degree in mathematics and physics from Yale University, as well writes daily about sports numbers on WSJ.com. He welcomes your comments at numbersguy@wsj.com.