Another Party Heard
Yet another study of partisan labeling, this one from the conservative
Media Research Center
, in an effort to redeem Bernard Goldberg's claim that the media label conservatives
more often than liberals.
The MRC looked at the use of the words liberal and conservative
in five years' worth of network news, culling out nonpolitical
uses of the terms (e.g., "a conservative estimate"), references to foreign
politics, and duplicate records. They then determined that the word conservative
is used about four times as frequently as liberal, a result
they trumpet as showing that "reporters are actually four times more likely
to label conservatives than liberals."
Not so fast -- the study actually proves nothing of the sort. Iindeed, it
proves nothing at all -- the MRC has cooked the books in a way that
even an Arthur Andersen accountant would blush to own up to.
For one thing, the MRC study didn't actually look at political labeling as
such, but merely the uses of the terms conservative and liberal
in their political senses. That means that they included not just
phrases like "conservative Senator Jesse Helms" but sentences like "Hard-core
conservatives have created a new verb, ‘Borked,’ after 1987 Supreme Court
nominee Robert Bork." But the latter are obviously irrelevant to the claims
made by Goldberg and others. After all, if the media merely said things like
"conservatives did such-and-such" more often than "liberals did such-and-such,"
it wouldn't suggest a liberal bias -- not unless, in the familiar paranoid
style, you assume that anyone who is talking about you must be saying something
bad. (By that same logic, you could argue that the fact that the press mentions
Sharon more than Arafat demonstrates its anti-Israel bias.)
In fact the MRC admit that most of their examples were of this generic type
-- itself enough to invalidate the study -- but they are curiously diffident
about giving an exact breakdown of how many instances of liberal and
conservative were actually used to label a specific politician
or group. And given that the MRC study didn't bother to separate out the
uses of conservative and liberal that actually function as
labels, they weren't in a position to answer the central question in all
these studies: what's the relative frequency with which such labels are applied
to politicians on either side?
What we want to know,
that is, is what the odds are that a given liberal or conservative
politician will be given a partisan label. But to make that determination,
you have to tabulate labels as a proportion of overall mentions of
the names of the people in question -- the procedure followed both in
my own study
and in a study of labeling by conservative blogger
. After all, the fact that the label conservative apappears more frequently
than the label liberal is meaningless by itself if the overall mentions
of conservative and liberal groups and individuals are not proportionate.
If Jesse Helms and Paul Wellstone are both labeled ten percent of the time
but Helms is mentioned five times as often as Wellstone, then you would expect
to find five times as many labels for Helms as for Wellstone.
And in fact it's clear that liberal and conservative politicians and groups
are not mentioned with equal frequency in the press. In Boyd's survey,
conservative politicians were mentioned overall more than two-and-a-half
times as frequently as liberals, which was pretty much what I found in my
study. And my study and
have showed that groups like the Heritage Foundation are mentioned four times
as frequently as liberal groups like the ADA. By failing to correct for these
differences, the MRC study stacked the deck -- it turns a discrepancy in
the overall number of mentions of liberal and conservative politicians
into a specious discrepancy in the frequency with which they are labeled.
Once again, the press are being charged with a liberal bias
because they mention liberals less than they do conservatives.
Is MRC simply too dim to understand this? Not likely. In fact the MRC has
used proportional counts in
, when it seemed convenient to do so. And they were well aware of both my
study and Boyd's, which appropriately used proportional counts. (To his credit,
Boyd himself has pointed out the
limitations of the MRC method and agrees with these criticisms.) So they knew what the correct procedure was, but didn't want to use it. Or what's equally likely, they actually did the right sort of analysis and then decided not to report it, since it didn't produce the results they were after.
One further point that MRC somehow didn't get around to reporting is how
often labels are used in the abstract -- are politicians and groups labeled
one percent of the time, five percent of the time, twenty percent of the
time? The omission is particularly significant because in
MRC pooh-poohed my own study on the grounds that the overall proportion
of labelings was very low:
In fact, Nunberg’s
"30 percent" gap was between how the liberals were labeled 3.78 percent of
the time and the conservatives were tagged 2.89 percent of the time. If the
MRC ever did a study which found that kind of puny difference we wouldn't
claim a 30 percent disparity. We'd say the media basically hardly ever do
x or y. So, if you buy Nunberg’s numbers he only found that newspapers hardly
ever label anybody, not that liberals are labeled significantly more often.
Yet when they actually did get around to doing that study, this particular Post-It seems to have slipped off their refrigerator door -- they somehow neglected to report what the overall proportion of labeling was. Why am I not surprised?
Added 6/30: One other thing that occurs to me: If the study doesn't bother to compute the proportions of liberals and conservatives who were labeled, it does at least allow us to infer just how frequently network TV labels politicians overall. The MRC looked at five years worth of broadcasts from the three major networks -- that is, at a total of about 1875 hours of network news (= three networks times five half-hour broadcasts per week for five years). They found a total of 924 uses of liberal and conservative as political terms, of which they admit that "most" were not labelings of individual politicians and groups. If, say, a third of the uses did involve such labelings, that means that the labelings occurred at a rate of about one every six hours of broadcasting on each network. That is, politicans were labeled by the networks at a rate of one every two-and-a-half weeks. And they don't call that "hardly ever"?
Added 6/30: In an email to the Washington Times in response to my criticisms, an MRC spokeman says: " Only someone with absolutely no first-hand knowledge of the ABC, CBS and NBC newscasts could suggest that conservatives were discussed four times more often than liberals." In other words: "Um, well, we didn't actually count the disparities in mention between liberals and conservatives."
Well, let's see if we can help. In one of the few claims that does in fact give enough information to be checked proportionately, the MRC study reports that the conservative label is applied to Supreme Court justices 49 times while the liberal label is used only 24 times, a two-to-one discrepancy. But now consider how often the names of justices on both sides have been mentioned on NBC news broadcasts over the past five years, using figures from the same Nexis database that the MRC claims to have used:
Mentions of liberal justices: Breyer (8), Stevens (16), Ginsburg (7), Souter (7): Total mentions: 40
Mentions of conservative justices: Rehnquist (104), Thomas (40), Scalia (50), Kennedy(14), O'Connor (40). Total mentions: 248.
Ratio of mentions of conservative to mentions of liberal justices: 6.2 to 1.
Ratio of total number of conservative labels to total number of liberal labels applied to justices: 2 to 1.
In other words, if we take the NBC figures as roughly representative of the networks as a whole, the MRC study shows that liberal justices are proportionately labeled three times more frequently than conservatives are. Bias indeed.
(With a six-million dollar budget, can't these guys hire a statistician?)