Yet another study of partisan labeling, this one from the conservative Media Research Center , in an effort to redeem Bernard Goldberg's claim that the media label conservatives more often than liberals.

The MRC looked at the use of the words liberal and conservative in five years' worth of network news, culling out nonpolitical uses of the terms (e.g., "a conservative estimate"), references to foreign politics, and duplicate records. They then determined that the word conservative is used about four times as frequently as liberal, a result they trumpet as showing that "reporters are actually four times more likely to label conservatives than liberals."

Not so fast -- the study actually proves nothing of the sort. Iindeed, it proves nothing at all  -- the MRC has cooked the books in a way that even an Arthur Andersen accountant would blush to own up to.

For one thing, the MRC study didn't actually look at political labeling as such, but merely the uses of the terms conservative and liberal in their political senses. That means that they included not just phrases like "conservative Senator Jesse Helms" but sentences like "Hard-core conservatives have created a new verb, ‘Borked,’ after 1987 Supreme Court nominee Robert Bork." But the latter are obviously irrelevant to the claims made by Goldberg and others. After all, if the media merely said things like "conservatives did such-and-such" more often than "liberals did such-and-such," it wouldn't suggest a liberal bias -- not unless, in the familiar paranoid style, you assume that anyone who is talking about you must be saying something bad. (By that same logic, you could argue that the fact that the press mentions Sharon more than Arafat demonstrates its anti-Israel bias.)

In fact the MRC admit that most of their examples were of this generic type -- itself enough to invalidate the study -- but they are curiously diffident about giving an exact breakdown of how many instances of liberal and conservative were actually used to label a specific politician or group. And given that the MRC study didn't bother to separate out the uses of conservative and liberal that actually function as labels, they weren't in a position to answer the central question in all these studies: what's the relative frequency with which such labels are applied to politicians on either side?

What we want to know, that is, is what the odds are that a given liberal or conservative politician will be given a partisan label. But to make that determination, you have to tabulate labels as a proportion of overall mentions of the names of the people in question -- the procedure followed both in my own study and in a study of labeling by conservative blogger Edward Boyd . After all, the fact that the label conservative apappears more frequently than the label liberal is meaningless by itself if the overall mentions of conservative and liberal groups and individuals are not proportionate. If Jesse Helms and Paul Wellstone are both labeled ten percent of the time but Helms is mentioned five times as often as Wellstone, then you would expect to find five times as many labels for Helms as for Wellstone.

And in fact it's clear that liberal and conservative politicians and groups are not mentioned with equal frequency in the press. In Boyd's survey, conservative politicians were mentioned overall more than two-and-a-half times as frequently as liberals, which was pretty much what I found in my study. And my study and others have showed that groups like the Heritage Foundation are mentioned four times as frequently as liberal groups like the ADA. By failing to correct for these differences, the MRC study stacked the deck -- it turns a discrepancy in the overall number of mentions of liberal and conservative politicians into a specious discrepancy in the frequency with which they are labeled. Once again, the press are being charged with a liberal bias because they mention liberals less than they do conservatives.

Is MRC simply too dim to understand this? Not likely. In fact the MRC has used proportional counts in other studies , when it seemed convenient to do so. And they were well aware of both my study and Boyd's, which appropriately used proportional counts. (To his credit, Boyd himself has pointed out the limitations of the MRC method and agrees with these criticisms.) So they knew what the correct procedure was, but didn't want to use it. Or what's equally likely, they actually did the right sort of analysis and then decided not to report it, since it didn't produce the results they were after. 

One further point that MRC somehow didn't get around to reporting is how often labels are used in the abstract -- are politicians and groups labeled one percent of the time, five percent of the time, twenty percent of the time? The omission is particularly significant because in earlier comments, MRC pooh-poohed my own study on the grounds that the overall proportion of labelings was very low:

In fact, Nunberg’s "30 percent" gap was between how the liberals were labeled 3.78 percent of the time and the conservatives were tagged 2.89 percent of the time. If the MRC ever did a study which found that kind of puny difference we wouldn't claim a 30 percent disparity. We'd say the media basically hardly ever do x or y. So, if you buy Nunberg’s numbers he only found that newspapers hardly ever label anybody, not that liberals are labeled significantly more often.

Yet when they actually did get around to doing that study, this particular Post-It seems to have slipped off their refrigerator door -- they somehow neglected to report what the overall proportion of labeling was. Why am I not surprised?

Added 6/30: One other thing that occurs to me: If the study doesn't bother to compute the proportions of liberals and conservatives who were labeled, it does at least allow us to infer just how frequently network TV labels politicians overall. The MRC looked at five years worth of broadcasts from the three major networks -- that is, at a total of about 1875 hours of network news (= three networks times five half-hour broadcasts per week for five years). They found a total of 924 uses of liberal and conservative as political terms, of which they admit that "most" were not labelings of individual politicians and groups. If, say, a third of the uses did involve such labelings, that means that the labelings occurred at a rate of about one every six hours of broadcasting on each network. That is, politicans were labeled by the networks at a rate of one every two-and-a-half weeks. And they don't call that "hardly ever"?