By Corey Cook
I am writing in regard to Reed Nelson’s story “’Poll’ showing 73 percent approval for Mayor Lee was flawed.” As one of the two authors of the survey, I am deeply disappointed in the many insinuations in the article and the author’s cavalier abandonment of evidence or reason in order to make his politically expedient, but otherwise inane, point.
In fact, the author is so quick to dismiss the findings of the study, which is based upon accepted methodology, and which had nothing to do with mayoral approval scores, that he actually misses the entire thrust of the study – that voters in San Francisco are deeply ambivalent about the current environment, concerned about the affordability crisis, and not trusting of local government to come up with a solution.
You’d think the Bay Guardian might find that an interesting subject. Under a previous editor I have little doubt it would have. Instead, the author mind numbingly asserts that the mayor’s approval rate – a largely irrelevant number – is clearly overinflated and the survey must then be “bogus” (meaning fake or phony). While other scholars might find the popular characterization of their work as “fake” somewhat amusing. I do not.
The author relies on two main sources to claim that an on-line panel survey is “bogus”, the New York Times “style guide” and the “website publication” of Southeast Missouri State University Political Scientist Russell D. Renka, who is neither a survey researcher nor a political methodologist, and who does not seem to have published anything in this field (or even in political science based on his on-line vita), but who does seem to have a fairly robust home page that includes cute photos of his grandkids.
It’s not the kind of “source” that I would utilize to deride another academic’s work as “bogus”, and I could suggest some other (actual) publications to consider, including Harvard political scientist Stephen Ansolabehere’s peer reviewed article in Political Analysis titled “Does Survey Mode Still Matter?” from 2011 that compares national surveys fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national RDD sample of landlines and cell phones), and by mail (using a national sample of residential addresses).
The authors of that study conclude that “comparing the findings from the modes to each other and the validated benchmarks, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.” But unfortunately that peer reviewed publication by a Harvard political scientist seems to contradict the simple assertion that a survey result the author doesn’t like must be phony.
Let me say that I don’t considered this issue “settled” in the scholarly community, but it is far from the case that serious on-line panel surveys ought to be derided as “bogus.” My preference would be to do a 1,200 person phone survey. If the Bay Guardian would like to commission such a survey, I would enjoy working with you on that project. But given the various cost limitations that preclude such a robust research design, this is not an altogether bad alternative.