Promotion, not research

Web-site polls are not a dependable gauge of opinion

In this most political of seasons, you may be tempted to post polls on your station's Web site. There are many questions you could ask: "Gore or Bush?" This senatorial candidate or that? "Should Bobby Knight have been fired?"

But Alison Schafer, an assistant professor of journalism at American University in Washington, cautions that Internet polls aren't "research." They are "promotions."

"TV stations' polls are thought to be 'unscientific,'" says Schafer, who has 11 years experience in local and bureau-based TV news. "I think that's the point: They're not meant to mimic Gallup; they're meant to provide viewers/Web-site users with a sense of being counted, of being important.

"The polls are another way to promo the station, which leads to more-fundamental questions about how news is being defined on the Web," she continues. "I think the Web-site polls are stations saying, 'We care about you. We're not really sure this poll reflects reality-heck, polls change all the time anyway-but we're interested in you and what YOU think.'"

She advises, "Stations should promote Web-site polls as a viewer's chance to weigh in."

But, she cautions, don't think they are anywhere near an accurate reading of the electoral sentiments in your metro area. Heck, they are not even a dependable gauge of the electoral sentiments of your viewership.

Seemingly programmed from birth with the empiricism gene, "regular" pollsters hate online polls. Don't talk to them about your online polls' "promotional" value. In their view, just the fact that you dare to call a Web-based survey a (shhhh) poll besmirches a sacred concept.

"The whole practice is not respectable," huffs Warren Mitofsky, president of polling organization Mitofsky International, New York, and a member of the Polling Review Board of the National Council on Public Polls, Hackensack, N.J. "Do not do Web-based surveys. They are misleading."

For pros like Mitofsky, it is a matter of principle. "Surveys should be held to the same standard [of] reliability as other news broadcasts by these news organizations," he maintains.

"If I cannot talk [stations] out of using Web surveys," he says, "then, after they say [on their Web site] that they are not scientific, they should also say, 'The opinions expressed in this poll do not represent the collective views of our community.'" (The world necessarily seems to be missing from Mitofsky's vocabulary.)

He maintains that stations that offer such disclaimers still go on to "discuss the results as though they were generalizing to the community."

Consider yourself scolded. But you are going to poll anyway, right?

"As a promotional tool, Web-site polls are great," says Schafer. "Only people with time and interest do them, and they in turn feel loved and important. That's what's good about the Net: Stations' polls are tailored to those who care; the rest of us can ignore them.'

"The rest of us" refers to a pretty large chunk of your viewership. Experts have long said that the primary contributor to online polling inaccuracy is the digital divide: Not every potential respondent to a phone poll has Internet access. With Internet penetration passing 60% of adults in some markets, though, that notion is starting to lose some of its relevance.

It seems that there's a deeper flaw in online polls: There's likely to be a disproportionate number of "true believers" in the respondent base.

Harris Poll Chairman Humphrey Taylor says that, while it is true that those who are on the Internet and will fill out a poll are likely to be younger and more educated than your overall viewership, the main biases are "behavioral and attitudinal."

By "behavioral," Taylor means that, because responding to a poll requires an element of action, results skew toward "doers." "Attitudinal" suggests a tilt toward those with "slightly more cynicism and skepticism" than the general public at large.

"We're just beginning to learn to statistically correct for these biases," Taylor told me recently. "We need to apply demographic weights and propensity weights."

If you still have your heart set on doing online polling, he suggests a "0 to 10, strongly disagree to strongly agree" model. In such surveys, people lean toward responding somewhere in the middle, somewhat muting the skeptical-responder bias.

Russell Shaw is a veteran Internet and broadcast-industry author/ journalist based in Portland, Ore., and can be reached by e-mail at His column appears regularly.