The post-debate poll put up this afternoon had predictable results- Barack Obama won with 37%, followed by John Edwards with 27% and Hillary Clinton with 12%. Of course, that tells me a lot more about candidate loyalties than it tells me about the actual debate performances. People vote for their candidate.
Or do they?
Well, mostly, they do. But there's a few people in there voting for the best performer, not their favorite candidate. If only there were a way to control for the sample bias and just look at the people who voted for the best performer...
Well, there is.
We took a straw poll here about 24 hours before this debate, which tells us ALMOST EXACTLY what the sample bias is (assuming that a representative sample voted in each poll). Controlling for the sample bias, it's clear that Hillary won the debate by "outperforming" her straw poll numbers with her debate poll numbers. Obama performed above expectations, but not as well as Hillary, and Edwards, well.... not so good.
Follow me for the math and anguish.........
OK, now for the tedious stuff. Let's examine this simple table.
No spitballs from the back, now, class.
To compare the proportion each candidate got in the debate poll vs. the straw poll, first we list the candidates (sorry, Dennis, your votes in the straw poll go in "Other" because you weren't in the debate). The raw number of votes for the debate poll is listed, then the percent of total debate votes for each candidate.
Now, repeat for the straw poll yesterday. As you can see, John Edwards got 39%, followed by Barack Obama at 30%, followed by Hillary Clinton at 8.5% (rounded to 8 on the report).
Eyeballing the data, we can see that Hillary and Barack did BETTER than expected if people had just voted for their candidate in the debate poll, and John Edwards did WORSE (he slid from 39% to 27%). Now, what's the best way to quantify this?
I used ratios. I created a ratio measure, which is basically the ratio of the proportion of the DEBATE vote a candidate got to the proportion of the STRAW POLL vote that that same candidate got. That's the column on the right. Numbers above one mean the candidate got proportionately MORE of the debate vote than their support here on dKos would be expected to generate. Numbers BELOW one means some of that candidate's own supporters voted for another candidate in the debate poll.
Forget Biden, Richardson and Dodd. The law of small numbers means that proportions close to zero are more susceptible to instability. Put another way, they could quadruple their support and still be below ten percent here. Their ratios also reflect the debate structure- they get a lot more time to speak than their voter support would indicate. Their ratios are interesting, but not comparable to the three leading candidates.
Of the three leading candidates, Hillary Clinton (ratio = 1.507) performed the best, gathering approximately three votes in the debate for every two votes she could be expected to get based on candidate support alone. Put another way, one-third of her votes in the debate poll were from non-supporters.
Barack Obama (ratio = 1.246) also did well, getting approximately five votes in the debate poll for every four he could be expected to get based on candidate support alone. Every four of his supporters voting in the debate poll were joined by a fifth non-supporter saying that he had won.
But poor John Edwards (ratio = 0.678). One out of three of his supporters deserted him. Not a good day for John.
POSSIBLE BIASES IN THIS METHODOLOGY:
- Sample size. Technically, this isn't a sample of the votes, It's ALL the votes. It's a sample of the universe of kossacks here. But the sample sizes are HUGE - over 5000 for the debate poll, and over 15,000 for the straw poll. No problem there. I'll leave it up to the dKos poll watchers to tell me about any systematic bases that may creep into the process ;-)
- Kucinich- sorry, Dennis. The "Other" categories are almost the same size (13.90% vs. 14.57%). Not a factor, unless the 8% of the dKos universe were somehow skewed in their distribution of votes to debate participants. My guess is that a lot of Kucinich voters stayed home on debate poll day.
- Numbers creep - people are still voting in the debate poll, so these numbers are from about 7 PM eastern and will be lower than the ones you see on the debate poll. There's still time to go stuff the ballot box, folks!
- I didn't do a test of significance. So sue me. Drum me out of the corps of statisticians. Tear that pocket protector off my shirt. Break my slide ruler over your knee.
- People ask me, "Well, HYPOTHETICALLY, what would happen if one were to stuff the ballot box?" Depends on which one you stuff. If you (hypothetically) stuff the straw poll ballot box and forget to stuff the debate poll ballot box, your candidate would look like dog meat in this analysis. Conversely, if you (hypothetically) stuffed the debate ballot box but not the straw poll, you'd look great. Wheels within wheels...