« General Election Cattle Call, August 22 | Main | Kerry Leading in 14 of 16 Zogby States »
Monday, August 23, 2004
Is Zogby's Interactive Poll Wide Open for Fraud?
Posted by DavidNYCSome industrious research has indicated that Zogby's "Interactive" polls - which are conducted online - don't vary materially from other, traditional polls. However, I was under the impression that Zogby ran these polls similarly to the way Nielsen does its TV ratings - that is, he uses a pre-selected, clearly-defined group of people who have agreed to participate.
But it seems that his online polls may be wide, wide open - at least, that's what this link would suggest. Maybe everyone knew this all along, and I'm just a bit late for this particular clue train. But with this link (just now?) getting play in a DKos diary, it seems to me that Zogby is in danger of having his sample population badly manipulated.
Perhaps this link is actuall for some other project, or perhaps Zogby has some very clever way of sussing out the phonies and jammers. But nonetheless, I still have new worries about the validity of these polls. Any thoughts?
Posted at 12:15 AM in General | Technorati
Comments
I certainly hope it's for another project, because the only cross-check possible in the questions I answered was that the state should correspond to the zip code.
I did try to answer it a second time (purely in the interest of seeing what would happen) and was stopped from doing so.
Also, note that the Daily Kos link you provided is broken in some way, and I couldn't find the story you referred to by using their search function. (I'm not sure that it searches the diaries.)
Posted by: Ed Fitzgerald (unfutz) at August 23, 2004 01:48 AM | Permalink | Edit Comment | Delete Comment
However, this description on the Zogby website of the questions asked in the battleground states poll is a pretty exact match for what I answered:
Questions that will be asked:
- If the election for president were held today, for whom would you vote -Republican George W. Bush or Democrat John Kerry?
- If the election for president were held today, for whom would you vote - Republican George W. Bush, Democrat John Kerry, or Independent Ralph Nader?
- Overall opinion - George W Bush
- Overall opinion - John Kerry
- Overall opinion - Ralph Nader
- Overall, how would you rate President Bush's performance on the job?
- Do you think President Bush deserves to be re-elected, or do you think it is time for someone new?
- US Direction
- What are the two most important issues facing the country today? If there were another major terrorist attack in the United States, who would you prefer to have as president - George Bush or John Kerry?
The only question I don't recall answering is the last one, about who I would prefer as president in the event of a terrorist attack. Otherwise, the survey I answered was the same.
Could it be a control group?
In the e-mail that had the link to the survey, Zogby wrote:
Zogby Interactive Surveys (ZIS) will allow you to voice your opinions regarding political issues and current events. ZIS combines the latest trends in technology with Zogby International���s record of success and technical expertise in field of public opinion research.
Posted by: Ed Fitzgerald (unfutz) at August 23, 2004 02:06 AM | Permalink | Edit Comment | Delete Comment
I remember reading about additional checks of 10 % (or was it 1%?) of the sample. In case Zogby double-checks indeed, and if they do that with a more accurate scheme (that is, checking for other demographic data), they should be able to statistically adjust their samples' data according to the errors found by way of the second stage check.
Furthermore, I figure they might be checking the users' IPs in order to sort out double answers given by users entering data with the same IP or, if they are more sophisticated, very similar data with similar or IPs that might look similar or schematic.
If they are somewhat sophisticated, they might also check if the IP corresponds with the ZIP-Code (or at least if it is credible).
Posted by: Bornheimer at August 23, 2004 02:40 PM | Permalink | Edit Comment | Delete Comment
I think I'm one of those being polled -- since I live in Pennsylvania. It was a voluntary thing on my part, but since then, they religiously re-contact me each week or so for a new survey, which changes a bit each time.
FWIW.
-P
Posted by: Paul L at August 23, 2004 05:14 PM | Permalink | Edit Comment | Delete Comment
Furthermore, I figure they might be checking the users' IPs in order to sort out double answers given by users entering data with the same IP or, if they are more sophisticated, very similar data with similar or IPs that might look similar or schematic.
If they are somewhat sophisticated, they might also check if the IP corresponds with the ZIP-Code (or at least if it is credible).
Unfortunately, the IP is no guarantee of geographic location, and there's the problem of services like AOL which use a proxy server.
Posted by: Ed Fitzgerald (unfutz) at August 24, 2004 01:24 AM | Permalink | Edit Comment | Delete Comment
Sounds like Zogby is a joke and should not be taken seriously!
Posted by: Pepe at August 24, 2004 06:54 AM | Permalink | Edit Comment | Delete Comment
I *know* that the IP isn't the greatest choice for data that should be checked - but the double checking (IF they do it, say, by an email requesting a phone number and a following phone call or whatsoever, even if only to get the raw polling data in line with the demographic data) and the possible error correction you can gain from data correction methods like that are far more important for getting a good result based on a sample designed from online replies than any purely technical stuff like IPs or whatever, I guess...
Posted by: Bornheimer at August 24, 2004 12:38 PM | Permalink | Edit Comment | Delete Comment
All polls are very suspect. Zogby's interactive polls maybe a bit more so.
Posted by: Clawed at August 24, 2004 02:46 PM | Permalink | Edit Comment | Delete Comment
I think Zogby's telephone polling is strikingly accurate.
On the other hand, any sort of internet polling should be looked at carefully.
Like electronic voting, the results are always suspect.
Posted by: Robert Marlye at August 25, 2004 11:00 PM | Permalink | Edit Comment | Delete Comment