Google Ads


Site Stats

Daily Kos Alleges Research 2000 Fraud

by: Crisitunity

Tue Jun 29, 2010 at 3:58 PM EDT


This is, needless to say, some pretty big news on the polling front. You probably recall that several weeks ago (after the Arkansas runoff, but apparently motivated primarily by 538's pollster rankings) Daily Kos severed its relationship with its pollster, Research 2000. Today, based on a study by three prominent statistics experts, Daily Kos is alleging that something is seriously amiss with Research 2000's polling, suggesting that the conclusions do not seem to reflect truly random polling. While the discrepancies seem most obvious in the weekly tracking polling and not state-to-state polling, Daily Kos has disavowed all numbers produced for it by Research 2000.

While the investigation didn't look at all of Research 2000 polling conducted for us, fact is I no longer have any confidence in any of it, and neither should anyone else. I ask that all poll tracking sites remove any Research 2000 polls commissioned by us from their databases. I hereby renounce any post we've written based exclusively on Research 2000 polling.

The gist of it is (as you might expect) best explained by Nate Silver, by excerpting the key graphics from the prepared report. The graphics show how R2K's weekly favorable numbers for Obama always seemed to move from week to week, usually by a small amount... which isn't indicative of a normal distribution. By contrast, Gallup's numbers form a very normal-looking bell curve, with a change of 0 being the modal amount of week-to-week change. The researchers who performed the poll also found discrepancies in rates of appearances of odd and even numbers (shades of Nate's takedown of Strategic Vision there).

Greg Sargent has details on the lawsuit that will be filed in short order by Daily Kos against Research 2000. For his part, Research 2000 head Del Ali tells TPM that he stands "unequivocally" behind every poll he produced, and is denying the allegations.

Needless to say, we at SSP have very much relied on the supposed quality of Research 2000's data, and will be watching further developments in this matter with great interest.
Crisitunity :: Daily Kos Alleges Research 2000 Fraud
Tags: , (All Tags)
Print Friendly View Send As Email

Daily Kos Alleges Research 2000 Fraud | 84 comments
Unbelievable
If the accusations are in fact true how did R2K think they were going to get away with it? Wow, just wow. Do they think people are stupid or something. I hope this Ali guy has some really good lawyers because he is going to need them. Daily Kos is like our sister blog so this is very disturbing. Does anyone know if Kos is going to poll anymore? Who will they use, PPP? I hope so.  

Proud member of the Indiana Democratic Party from IN-9.  
Quote
"Soon, we'll have a new pollster (or pollsters) to work with, helping us to fulfill our vision of surveying races and issues that are often overlooked by the traditional media and polling outfits."
[ Parent ]
Good
Any ideas who. What about Rasmussen? JK. Seriously what about Anzalone Liszt? It is a smaller outlet, and they were dead on in 2008.  

Proud member of the Indiana Democratic Party from IN-9.  
[ Parent ]
I want PPP
PPP is already fairly prolific, but they have been for the most part fairly reliable and Tom Jensen is a guy I respect a lot. I'd like to see more of their work.  
[ Parent ]
I think that depends on if they want live
interviews. Does PPP do live interviews as an option for their clients?
[ Parent ]
No way, that wouldn't add anything new. A different pollster...
...would be better.  And I want live caller.  We already have 3 different robopoll firms working prolifically out there.  I want more live caller polls for comparison.

What I want is some diversity, and PPP doesn't add any because they already let fans decide by vote on where to poll week after week after week.  I want more volume so we can have additional data to more intelligently weigh the state of various races.

I'm sorry that R2K screwed DailyKos and all of us campaign junkies, but I've been missing DK's regular polling of races and hope they pick somebody and start again soon.

43, male, Indian-American, Democrat, VA-10

[ Parent ]
R2K may countersue
I wouldn't be so sure to jump on board with kos on the fraud allegations.

There's a big difference between being a bad pollster and a cheater/liar.

The quotes from R2K's lawyer
[ Parent ]
Agree
I'm taking a wait and see attitude on this one.
[ Parent ]
There's not so much to wait and see about
Anyone with statistics knowledge would realize that there is fraud in the polling results. They produced nearly statistically impossible results, well outside any expected range, and therefore, fraud should be suspected. If this case goes to court, Markos is going to have a fairly easy time winning the case.
[ Parent ]
Yeah
Right of reply is so un-American.
[ Parent ]
Lol
tend to agree. Let us see how the results play out.  
[ Parent ]
Did I say they could not reply?
I said the fraud is clear, so there is not much to wait and see about. R2K should and will respond.
[ Parent ]
Then why not wait and see what they have to say?
The evidence is persuasive but in my view not conclusive.
[ Parent ]
R2K has been given the opportunity to respond.
From Kos:
http://www.dailykos.com/storyo...

"Meanwhile, Research 2000 has refused to offer any explanation. Early in this process, I asked for and they offered to provide us with their raw data for independent analysis -- which could potentially exculpate them. That was two weeks ago, and despite repeated promises to provide us that data, Research 2000 ultimately refused to do so. At one point, they claimed they couldn't deliver them because their computers were down and they had to work out of a Kinkos office. Research 2000 was delivered a copy of the report early Monday morning, and though they quickly responded and promised a full response, once again the authors of the report heard nothing more."

(emphasis mine)

27, Democratic, IL-01

[ Parent ]
As a lawyer, I don't believe in
fairly easy winning in court.  Yes, you can have a strong case, but winning in court is never easy.
[ Parent ]
I should think that
bad pollster = incorrect results and
fraud = statistically impossible results.

I mean, obviously all we have is the memo written by the three guys and Nate's comment which pretty much validates them.  But what they're saying goes beyond bad polling.

[ Parent ]
According to Greg Sargent
Dkos's and it's lawyers have taken the initial statistical findings and brought them before several other expert Statisticians and have had it confirmed that the numbers do not seem to indicate natural polling. It seems like R2K might have gotten lazy on the extensive Week Tracking polling, which is, imaginably, expensive and big work, cheaper and easier to try and thoroughly fabricate such information, at least that's my best guess as to why they did it.  
[ Parent ]
It'd be really simple
to just start writing in numbers similar from the day before and just go, oh they changed by a point, you know, not much going on so that makes sense.
[ Parent ]
And of course they look at the news
and follow other pollsters, so they can correlate to bigger bounces. Of course they'd add a few points to his approval rating too, seeing as their client wanted a broader, more Democratic leaning electorate polled.
[ Parent ]
Surely does not help pollsters
There already is a perception that pollsters are full of it.  I am sure this does not help much.

It really does not surprise me.  The R2000 numbers have been highly erratic for months.

23, male, center-right cynical Republican, PA-7

I'll reserve judgement for now...
...since statistics can often show strange results.  I'd like to believe a firm that has such a compelling interest in producing accurate results for their clients wouldn't play with their numbers beyond the traditional massaging based on what they expect the turnout model will be.

Kos is a pretty smart guy so I doubt they'd make a move into the legal realm if they didnt think they had a firm footing.  Unless it's a paper lawsuit meant more to cover themselves and the negative publicity from having to distance themselves from their own pollster than to actually try to go to trial.

NY-13, Democrat. Blog @ http://infinitefunction.wordpr...

Your final line
Frankly, I had the same thought. Like I said, keeping an open mind.
[ Parent ]
There's really no point in suing
if all it is for is to cover themselves.  Lawsuits can be very expensive and a money suck - nothing something one does willy-nilly.
[ Parent ]
The thing about R2K is that they've
nailed some races where other pollsters have missed.  Take AK-Pres in 2008 as an example.  So it seems that his horcerace polling is honest but just shaky in general.  The evidence is only against his polling for the weekly favorable numbers, though Silver suggests that something was off with his tracking poll (even though Silver told everybody when it first came out in September 2008 that people shouldn't be so quick to question it even though it was  more favorable to Obama/Biden at that time than other pollsters).

It's hard for me to believe that this guy would be honest with his horcerace polling and then just cheat on the weekly tracking.  I guess you can argue that there is no accountability for weekly tracking since it cannot be tested until 2012, so he saw an easy way to cheat.

Nate Silver had concerns and apparently
e-mailed the pollster.com people, as he noted here:

http://www.fivethirtyeight.com...

Ipsos is now polling state races?
They show Boxer leading Fiorina 45-41 and Brown leading Whitman 45-39.
Link? (nm)
nm

43, male, Indian-American, Democrat, VA-10
[ Parent ]
What does nm mean?
[ Parent ]
NM = No Message
All he's saying is that what he said in the title is the entirety of his post.

Politics and Other Random Topics

24, Male, Democrat, NM-01, Chairman of the Atheist Caucus, and Majority Leader of the "Going to Hell" caucus!

[ Parent ]
Crisitunity
You are a statistician too, are you not, what are your thoughts and opinions on all this?  
I'm not being evasive
when I merely say "I'm reserving judgment for now." There's always an alternate hypothesis for why numbers turn up the way they do, and I'm waiting to see if Research 2000 offers one up.

Also, amusingly, this may be the first time I've ever been called a statistician. I have taken a few graduate-level stats courses, enough to correctly throw around words like "modal," but it's far from what I do professionally.

[ Parent ]
Fan hit by the proverbial at 538

I was going to say "huh."
But I then I got it.  I'm a bit surprised by the C&D.
[ Parent ]
Fraud allegation
Fraud is a serious allegation and Kos is a Northwestern Law school grad (not sure if he is licensed to practice law or even ever practiced law), so I'm guessing he's thought about what has happened.

To a newbie lawyer like me the fraud case may be harder to prove than defation; however, if it turns out he can't prove scienter, can R2k turn around sue him for defamation?

Obviously this is a legal question, but can you imagine a court case where the evidence basically centers around comparing polls with election results?  

Indepedent/Lean D. Dude.
All 5s (now TX-5; frmly VA-5 and CA-5)  

I can imagine a court case
Where dueling experts bamboozle everybody with statistical variation and probability that nobody understands!
[ Parent ]
Evidence
I think the evidence will actually center on records of polling (phone calls, collected data, etc...), as well as the statistical anomalies from the polling. I am not sure if the accuracy of the polls really matters much in the end for the case. The question here is whether R2K produced real polls and provided real data to DailyKos or whether the polls were fabricated. Even if the R2K's polling was completely accurate, the data could still be fraudulent if the results were produced by non-polling methods or other methods not stipulated in the contract with DailyKos. Any inaccuracy in R2K polling would likely only add to the other more chief pieces of evidence.  
[ Parent ]
couple things
1. My understanding is that Kos went to Northern Illinois for undergrad and Boston University for law school.

2. My understanding of prospective summons and complaint from Adam Bonin (Kos's attorney) is that the lawsuit will be based on an action for breach of contract (with all the ancillary allegations based on the breach of contract claim i.e. fraud, unjust enrichment, etc.)

[ Parent ]
R2K issues "cease and desist" letter to 538
So this is the R2K counterattack, ref http://www.fivethirtyeight.com...

Copy of cease and desist letter included in the link.

Nate says no:

I emphatically stand behind any statements I have made about Research 2000, and will be constrained by nothing other than my common sense and my professional integrity in any comments I should elect to make about Research 2000 in the future.

Nasty
This is interesting to say the least.

23, male, center-right cynical Republican, PA-7
[ Parent ]
Research
2000 is like a wounded animal right now.

19, Male, Independent, CA-12
[ Parent ]
The evidence against R2K seems very thin IMO

I'm gonna have to agree with this
I think R2K is guilty of working within a very specific demographic framework set forth, ironically enough, by Markos Moulitsas and the folks over at Daily Kos. I think that R2K may have overcompensated for the flawed, overly white and conservative likely voter pools used by other pollsters say, Rasmussen in particular, and instead oversampled women and minorities in many cases. I'm surprised at the extent to which Markos has repudiated R2K in recent weeks. IMHO, it seems Markos went overboard when Halter lost in Arkansas. The final result was in the margin of error of R2K's polling; in addition R2K was attempting to poll a runoff in a Democratic primary in a Southern state, factors which on their face may not seem especially anomalous but are in fact difficult to properly quantify.

Also, Markos was using R2K quite heavily on a daily and weekly basis to publish polls many different kinds of polls. I think a combination of overwork and a lack of personnel and resources within R2K contributed to the shoddy, but not necessarily fraudulent, work.

[ Parent ]
It really doesn't
Basic tests were performed to determine the "naturalness" of the crosstab numbers, and found they had a likely hood of 1 X 10^28 of happening, the Even + ODD correlation. The numbers don't fit any statistical models either. There seems to be an incredibly strong case and R2K has had a while to repudiate it, or, stop the whole thing from happening by providing the full and detail records of the polling in their defense when asked to several weeks by Dkos or in the aftermath of this move.  
[ Parent ]
Basic, shmasic
Also from the report: "This does not tell us whether there was a minor 'adjustment' to real data or something more major."

Again, I think R2K was providing real, palatable information to the Daily Kos community, and I believe the firm consistently "adjusted" its data to provide for uniformity. I don't think there was fraud as such--I think that the information was always more or less right and based on actual polling, it's just that R2K's standard methodology was regularly adjusted internally to allow for sensitivity to specific demographic groups, especially the so-called "non-traditional" voting groups which Markos is expecting to supplant white (male), conservative voters.

In other words, I think R2K regularly oversampled certain groups to compensate for discrepancies in what it believed the end result should be.

In other words, I think they were trying to give Markos what he wanted, namely evidence that a left-leaning electorate of young people, women and minorities is slowly emerging. For example, I think that R2K would survey 1,000 women when the number was probably more like 1,100 or 1,200 women in order to produce numbers which would be palatable to the Daily Kos community and then say they surveyed 1,000. Fraud in this case would still be difficult to prove because the evidence was probably expurgated when the polling was completed.

What Markos was buying from R2K was a la carte polling. Why would I continue paying someone money if they're not showing me numbers which bolster my cause, i.e., that Obama is more "favorable" than he is "unfavorable" and that Democratic candidates in horse-race polling are still performing quite ably in a difficult environment.

It's only when Halter lost and Nate Silver did his "hit piece" on 538 that Markos got embarrassed and repudiated R2K.

It doesn't surprise me that R2K shot a cease and desist over to 538.

[ Parent ]
Your defense of R2K is very weak......
That one sentence from the report doesn't exculpate R2K in any way, shape, or form.  The report makes clear there are numerical patterns in the polling data that doesn't correspond to random sampling, and a bunch of respected experts have corroborated the numbers are fishy.  At Talking Points Memo, Josh Marshall took down R2K data from his poll aggregation system based on discussions with polling experts corroborating the suspiciousness of the data.

To date not one expert has proferred a defense of R2K data.

Nor did R2K ever present Markos with the underlying raw data he requested.

There's plenty to conclude R2K is hiding shenanigans.  And if those "shenanigans" were nothing more than twisting data to conform to Markos' politics, they would've said so.

On the contrary, there's no evidence whatsoever to support your suppositions.

43, male, Indian-American, Democrat, VA-10

[ Parent ]
I couldn't disagree with you more
When one looks at the number of different polls which Markos was commissioning in a number of different states, sometimes over a very short span of time, one could only take away that R2K was doing the "best it could" and in some cases may not have allowed necessarily for the nuances of specific states, as some other polling outfits do.

PPP and Mason-Dixon are the only other pollsters which work in a number of different states, and when I look at their numbers, I don't think they push leaners the way R2K does.

Also, I think all of this proves that R2K made minor, inadvertent adjustments to its data internally on a consistent basis, which, again, has more to do with the "product" it was delivering than the actual "substance". In other words, I think they were more concerned about the way the final numbers would look than defrauding anyone, per se.

For example, the "odd-even" correlation which keeps being discussed is "odd!", but I think it reflects more an internal idiosyncrasy than an intentional misleading. I think this because the "odd-even" correlation cannot be proven as outside the range of what is acceptable and normal. The "odd-even" correlation is best explained as a way to overcompensate for data which may not be fully reflective of the opinions of certain demographic groups. If I wanted to poll 10 blacks and only 7 responded, I may adjust my data to compensate for the other 3 non-responders.

[ Parent ]
That doesn't get at the point of the issue
The odd-even correlation has nothing to do with compensating for any disparity between population demographics and sample demographics. It's a question of how likely it is that, in their responses, the responses of men and women on approval/disapproval are correlated; and correlated such that in virtually EVERY case, both were either even or odd. If I was to take 10,000 polls of 1,000 people each, I'm not going to get such a result 99% of the time. As often as not the results should show a 44/51 or a 37/22 result, where the approve/disapprove split aren't obviously correlated. If, as you seem to claim, some element of Ali's weighting methodology resulted in him "massaging" the data and crosstabs to fit what he thought looked good, then it's still fraud, and the poll remains useless. Period.

More important and more damning is the Obama approval rating trends. To use a very basic example, if I'm gauging support for a candidate, my numbers will often fluctuate from week to week, but sometimes they'll also stay the same. In fact, a net change of zero will be the expected mean, barring earth-shattering changes. In polls with a large enough sample size (approximately >300), the net change in approval should thus approximate a normal distribution centered at or around zero.

What R2K's numbers showed is exactly what I would expect if one was actively trying to fake polling. If you think of how to mimic random shifts from an approval starting point, you'll go "oh, ok, a point this week, a point the other way next week, two points back next week..." If you're trying to fake randomness, you unconsciously reject the possibility of no change (approval at 46, approval at 46 again, huh, that doesn't seem random), though randomness should produce exactly that quite often. That R2K's numbers almost never showed a change of zero and likewise rarely showed a change larger than 2 (don't rock the boat, keep things consistent to avoid arousing suspicion) leaves no doubt in my mind that they never actually carried out randomly sampled polls. Or if they did, they were, to put it mildly, "monkey-fuck ridiculous."

[ Parent ]
I wouldn't call it "massaging"
I'd call it "extrapolating" based on the supposed voting preferences of a specific demographic.

In the case of Obama's favorability ratings, this would not be fraud; it would be supposition consistent with the mission of Daily Kos: to include the opinions of more and more "non-traditional" groups in politics.

[ Parent ]
Even if they "extrapolated" the data, it's still fraud
They are trying to pass off non-random data as being random, it doesn't really that their motives were pure, what matters is that they actually committed fraud.

And you can't just suppose you know how the remaining sample might vote in your final product, they don't have those extra people in the sample, extrapolating how they should behave defeats the whole point of polling in the first place (you get more information about how they might vote by actually getting more of them into your sample, period).

Politics and Other Random Topics

24, Male, Democrat, NM-01, Chairman of the Atheist Caucus, and Majority Leader of the "Going to Hell" caucus!

[ Parent ]
I disagree
I don't know the standard of fraud in this case in a legal sense, but in my mind it would be wholesale manipulation or creation of data as "fact."

There was no wholesale manipulation as I see it.

R2K simply made polite suppositions.

I thought we all knew that, seeing as how R2K's "favorability" ratings for Obama would be more than 5 to 7 points off from Gallup's "approval" ratings for Obama on any given day.

I thought--and I still think--Markos and R2K were setting their own narrative about the various "new" voting constituencies continuing to be happy with Obama in a way the traditional voting groups weren't.

[ Parent ]
Favorability is *not* the same thing as Job Approval
True, they are often highly correlated, but they are asking inherently different questions, as such they aren't compatible for that type of comparison. I think they should have asked straight-up approval ratings (or ask both of them) but there's nothing inherently unseemly about it either.

I love how you keep calling what R2K seemed to have done "polite suppositions" and "extrapolation" but all you really say is that Research 2000, instead of actually polling all 10 people (by your hypothetical), polled 7 people and substituted their own judgment for the remaining 3 (meaning that a full 30% of that data is, at best, not actual data, but an extrapolation of an extrapolation).

This, incidentally, raises a whole host of new problems, like how exactly they would come to such an extrapolation in the first place? If they simply extrapolated it based on the first 7 callers, then you've artificially lowered the Margin of Error for no good reason. If they took the data based on what other pollsters have shown, then they're stealing other data and trying to pass it off as their own. Are they trying to extrapolate it based off of actual voting patterns in the past? And all of these problems assume that they only extrapolated the numbers for one sub-sample, which is highly unlikely (which means that the entire sample is suspect because a large amount of it was likely not actually gathered).

And the general standard in civil court is not "beyond a reasonable doubt", it's "preponderance of evidence", all Daily Kos has to prove is that it's more likely than not that Research 2000 has breached the contract with Daily Kos, and it's really easy to prove even with your hypothetical (since it's really doubtful that Daily Kos agreed to allow Research 2000 to fudge any of its results).

Politics and Other Random Topics

24, Male, Democrat, NM-01, Chairman of the Atheist Caucus, and Majority Leader of the "Going to Hell" caucus!

[ Parent ]
It's not "fudging" or "massaging"
I believe R2K had Daily Kos' implicit approval to track certain demographics which heretofore have been under sampled.

In the case of favorability polling, I believe R2K thought it acceptable to overlap results from similar polls of its own. I don't believe they copied others' work. And I don't think they made things up out of whole cloth. Markos wanted R2K to replicate the WSJ/NBC tracking polls at a discounted price. R2K did the best it could with the resources it had.

[ Parent ]
You can't have "10" people polled in a sample if you didn't poll 10 people
You account for under-sampling by including more actual people from the target demographic, you don't just bring in people who were not part of this polling sample, period.

I have plenty of problems with Daily Kos, but pretending that your hypothetical situations are, in themselves, not wrong-doing or big polling no-nos just doesn't make a lot of sense.

Politics and Other Random Topics

24, Male, Democrat, NM-01, Chairman of the Atheist Caucus, and Majority Leader of the "Going to Hell" caucus!

[ Parent ]
Eh, if you know what they're going to say, what's the big deal?
n/t
[ Parent ]
Any honest adjustment doesn't remove the proof of random sampling......
You do a poll and adjust for a favorable turnout model or other things to produce a poll favorable to your client, you still ought to have all the signs of random sampling.  The distribution of numbers should still show that the sample was random.  None of your arguments explain why evidence of random sampling would be missing, as deliberate bias in results doesn't change the signs of randomness.

43, male, Indian-American, Democrat, VA-10
[ Parent ]
If all sub-groups are weighted and weighted again when combined, then...
I think R2K--especially with the favorability tracking polls--runs their numbers through a specialized formula (i.e. an algorithm) to account for day-to-day discrepancies, holes or missing data of specific sub-groups.

That's not fraud--that's delivering a product reflective of what is generally known, for example:

1. Black women would by-and-large not be disapproving of Obama.
2. Black men would by-and-large not be disapproving of Obama.

If I'm doing a daily tracking/favorability poll and I can't reach 5 respondents, using previous data based on that sub-group and factoring it into my latest poll is not fraud.

I don't know what to call it--we keep coming back to the term "fraud" because some of you are purists and would rather a poll not be completed than data from very previous polls be factored in on any given day.

[ Parent ]
Pass the popcorn
http://bluegrasspolitics.blogi...

Del Ali, president of Research 2000 in Olney, Md., said he could not respond to the specific allegations Tuesday and referred questions to his attorney, who did not return a call seeking comment.

"I can tell you, we're fine. What we're going to reveal, that will be the end of the Daily Kos," Ali said. "I can say, it has to do with people owing money."

Yeah
I assume he will try to take down Kos, whether or not there is anything to say. This won't end well for anyone, irrespective of fault.

I will say that if male and female opinions of Obama start out both even or odd on week one, the chances that they will move the same amount when subjected to the same political forces is not the same as the chance that two randomly selected quantities will end in even or odd digits. But two hundred something out of two hundered somthing plus two certainly does sound "monkey-fuck ridiculous."

Could they have done this: step one, use one set of data to determine overall movement of the electorate, step two, use a different set of data, or an unreliable subset of the same data, to test for gender disparity in the movement of public opinion? And then so consistently failed at step two, or so stubbonly assumed a null hypothesis against weak efforts to prove gender variation in the movement of public opinion, that they always reported "no difference" by gender in the movement of public opinion?  

This is going to end ugly, that's for sure....
There will be no winners...  Accusing a polling company of fraud is pretty serious stuff.  Any hint of malfeasance by the company would kill all of their business.  If R2K has an adequate defense, and it doesn't need a lot, then the damages from a defamation lawsuit would be immense!

Markos better have his ducks in a row.... theoretical testimony from some statisticians is not enough to beat a defamation lawsuit, nor is it enough to prove fraud.  He's going to need to find some good evidence in discovery.  If he doesn't, Markos could be in big trouble.

We'll see.  He hired an attorney before leveling the accusations.  I presume his lawyer knows what he's doing.

[ Parent ]
Seems like phone records might be pretty probative

[ Parent ]
I don't think R2K could sue Daily Kos for defamation
Wouldn't the R2K firm be a limited purpose public figure under NYT v Sullivan, if not a public figure? That means that R2K would have to prove actual malice, which is very difficult.
[ Parent ]
Without a smoking gun
I don't even know how anything can be proven in this.  Just because something stands near realtive impossibility in statistics does not (and cannot on its own) demonstrate fraud.

I'm also one to believe that usually when people commit fraud in their field of work, they either go for very easy stuff thats small potatoes and not that important (think the penny skimming scheme in the movie Office Space) or overly-elaborate schemes that no one could ever detect.

To me this seems like they took something overly easy to detect that was very important, which isn't usually a footprint one finds with fraud.

Unless an insider comes forward, I have no idea how you prove fraud in this instance.

I think they were giving Markos what he wanted
n/t
[ Parent ]
I actually read Nate Silver's article now
Probably should have before.  

Doesn't the gallup poll look alarmingly bell-shaped, like almost forced.  I find that more perplexing than the R2K shape.

Why exactly is no change week to week the expected mean anyways?  Do they ask the same people the same questions week after week or is it a different sample that is supposed to be the same makeup?  

[ Parent ]
Good point
One of the fun experiments they do in intro statistics classes is to create a bunch of randomly generated data following Gaussian distributions just to show the students how irregular they are in practice.  That being said, the R2K polling is quite anomalous.  You'd need to come up with an explanation for why zero change is less likely than expected.  

28, Unenrolled, MA-08
[ Parent ]
Nate has a new article up
http://www.fivethirtyeight.com...

We need a comparison with Rasmussen and Gallup because my recollection was that the former was even more consistent. The reason being strict partisan weighting.

[ Parent ]
I dont think there is any difference between...
proof of something and proof that something is statistically unlikely. Circumstantial evidence is often much more powerful than direct evidence. I do think it is worth hearing an explanation, though. Since there is no self-incrimination clause in civil cases, I think leaving the stats unexplained is effectively conceding the case. But what the hell do I know.
[ Parent ]
Please dont sue me

[ Parent ]
Too late I already filed the suit
The non-randomness of your posts is my evidence.  Maybe we'll be on Judge Judy.
[ Parent ]
I don't know...
Unfortunately in the past few years in America we've seen just about everything.  Things that never could happen did happen.

People don't need to be statitiscians to know that sometimes things cant be explained.  People are "wowed" by numbers though, they tend to fog the mind.

[ Parent ]
one over ten to the twenty-eighth is pretty wowie

[ Parent ]
So is
10% unemploymnet, Lehman going under, WTC bombings, etc.

My point is that weird things can happen.  People believe now more than ever that they can.  

So wacky coincidences is more than just a possible explanation, its something that can be played in people's minds.

Not that its right or wrong, just that these days we as a country are more likely to believe in wacky things happeneing than maybe we were 15 years ago.

[ Parent ]
I think what he'd likely sue on is breach of contract
Which, though I'm not lawyer, seems like it'd be pretty easy to prove. You state that R2K was contracted to provide randomly sampled polling data, including a weekly tracking poll. Use statistical evidence to demonstrate that, whatever numbers were received were not in fact based on random sampling.

If you can prove that the possibility of the results received occurring as a result of random sampling is vanishingly small (something that I think Grebner et al did quite nicely), I feel like that'd be your case right there.

[ Parent ]
If R2K can counter with how they did their polls
and show the raw data from individual samples, then they can counter.

I do not know if pollsters keep that kind of data for the record. Five years ago, I'd doubt it. Today, I am  amazed at the terabytes of data that's collected for major e-tailers on a daily basis.

[ Parent ]
Maybe
I think that the fact that they haven't provided this data as of yet, however, speaks to the fact that they DID feed DKos bullshit. Why would you refuse to provide data to a client when you claim that said data that would absolve you?
[ Parent ]
Are there privacy laws at issue?
Computer-based data collection is difficult enough.

IF that's the case (and I acknowledge that's a big IF, especially as IANAL), then such data may only be suitable for a courtroom in some sanitized form or forum.

[ Parent ]
No privacy issues, respondents' names and phone numbers easily withheld without...
...compromising any analysis of the data.  The only way the respondents' names and phone numbers are relevant is if one believes that data itself is falsified!

If you withhold a respondent's name and phone number, there's nothing else in the records that's subject to any privacy laws.

43, male, Indian-American, Democrat, VA-10

[ Parent ]
I have a degree in Statistics
And to be honest with everyone, I haven't done anything with that degree in the last 10 years.  I don't have recent experience to make a sound judgment on whether fraud or neglect was involved in these numbers.

At first glance, the analysis provided by Kos indicates that the numbers provided by R2K should not be relied on.  Upon further review, I feel that these numbers, especially on Obama's approval ratings, do not properly reflect normal regression over a period of time.  Specifically, the approval numbers lack the proper static that would reflect the electorate's opinion on Obama.  

That being said, I'm withholding judgment on whether R2K committed fraud.  I think it would be easier to prove that there was a level of neglect (not fraud, since neglect can be deemed unintentional) than fraud.

I'm keeping an open mind, but I'm not open to relying on R2K numbers in the past or present.

40, male, Democrat, NC-04

Revealing and helpful stuff from Chuck Todd & MSNBC "First Read" crew......
This is good stuff:

Link:  http://firstread.msnbc.msn.com...

Here's a little secret: Good polls are expensive to do, and if you're seeing a particular organization doing a slew of polls, you've got to ask: (1) how reliable are those numbers, or (2) where is the money coming from to conduct those polls? Nowadays, on the state level, we trust the polling we're getting from campaigns and state parties (although not necessarily those polls that are made public) more than the numbers we see from some non political polling organizations.

Our policy to you on state polling:  One policy we're going to institute ourselves to make sure you have an idea of everything that we know is this: When we report a public poll on the state level, it will be because we think those numbers are reflecting what we know is going on in the race. We'll let you know if a pollster has a good reputation in that state, has a good track record (because a good pollster in one state doesn't mean they know the nuances of another).

And, btw, First Read later approvingly cites the latest Quinnipiac OH-Sen poll and the new Reuters/Ipsos CA-Gov and CA-Sen polls, saying all those polls are consistent with private polling.

I've recently been gravitating away from viewing robopolls as reliably as live caller polls.  And the First Read blog post reinforces the point for me.  Chuck Todd said awhile back that if robopolls were so good, then campaigns would use them--and they don't.  I always figured the political media are privy to good private polling on both sides, and what they say about the state of a race reflects that.  It's a given that private polling is better than public polling because parties and candidates must have the most accurate data possible.  They make their most essential campaign decisions based on their own polling, unlike campaign junkies like us who are hungry for data merely to feed a personal hobby.

43, male, Indian-American, Democrat, VA-10

I really like Chuck Todd
For the reputation MSNBC has (which they STUPIDLY perpetuate by having Keith Olbermann and Rachel Maddow), Chuck Todd is extremely non-biased and is much better at that than some other prognosticators.  And unlike a few of the others, he doesnt talk out of his ass so he can have a story to write about.  And unlike Cilliza, he knows about the races he talks about (reading The Line is border-line pointless, especially if you follow this blog.)
[ Parent ]
Cilliza is worth reading b/c he gets private information......
Like political journalists generally, party and campaign people talk to him and share private info on the state of the races.  So his "Line" is worth something more than, say, if I did a race ranking.  That doesn't mean informed campaign junkies like us can't disagree with him and ultimately prove him wrong, but it does mean his rankings are worth something.

43, male, Indian-American, Democrat, VA-10
[ Parent ]
I don't see why this is surprising
R2K is even worse than Rasmussen, and unlike Rasmussen has no explanation for its flaws.
after I wrote this up
at Bleeding Heartland, Drew Miller (Bleeding Heartland founder, by the way) checked the results from polls R2K did for KCCI-TV in Des Moines this year and found the same even/odd pattern. On every question, the responses for men and women were either both odd numbers or both even numbers.

Mark Blumenthal's post on this yesterday is a must-read. He asked a forensic data guru to look at the analysis:

[Walter] Mebane says he finds the evidence presented "convincing," though whether the polls are "fradulent" as Kos claims "is unclear...Could be some kind of smoothing algorithm is being used, either smoothing over time or toward some prior distribution."

When I asked about the specific patterns reported by Grebner, et. al., he replied:

   None of these imply that no new data informed the numbers reported for each poll, but if there were new data for each poll the data seems to have been combined with some other information---which is not necessarily bad practice depending on the goal of the polling---and then jittered.

In other words, again, the strange patterns in the Research 2000 data suggest they were produced by some sort of weighting or statistical process, though it is unclear exactly what that process was.


Daily Kos Alleges Research 2000 Fraud | 84 comments

Copyright 2003-2010 Swing State Project LLC

Primary Sponsor

You're not running for second place. Is your website? See why Campaign Engine is ranked #1 in software and support among Progressive-only Internet firms. http://www.mediamezcla.com/

Menu

Make a New Account

Username:

Password:



Forget your username or password?

About the Site

Publisher: DavidNYC

Senior Editor: James L.

Senior Editor: Crisitunity

Contributing Editor: Jeffmd

Search the SSP archives:

Sign up to receive a daily e-mail whenever the SSP is updated:

SSP Resources

Presidential Vote by CD

SSP Document Collection

Race Tracker Wiki

Dave's Redistricting App

Blogroll

Daily Kos Elections
DemConWatch
Five Thirty Eight
Governing
Hotline On Call
Political Wire
Pollster.com
Prop*50
Public Policy Polling
The Thicket
Powered by: SoapBlox