LOOKING FOR MORE?

How do Americans feel about cap-and-trade legislation?

In recent weeks, two media pollsters reported results on the point. "Six in 10 Americans support a 'cap-and-trade' proposal to cut pollution," said the CNN/Opinion Research Corporation poll. Despite "growing public skepticism about global warming," the Pew Research Center found "more support than opposition for a policy to set limits on carbon emissions."

How accurately do these questions measure public opinion on cap-and-trade legislation?

To answer that question, you may want to consider how Americans answered another: "Some people say the 1975 Public Affairs Act should be repealed. Do you agree or disagree with this idea?"

As a well-informed reader of NationalJournal.com, you are probably inclined to wrinkle your brow and ask, "What's that?" For good reason: It never existed. But its fictitious nature didn't stop 34 percent from expressing an opinion when University of Cincinnati political scientist George Bishop and his colleagues asked a sample of Cincinnati adults that question in 1978. Bishop and other scholars have consistently replicated that finding using national samples and similarly fictitious or unknown legislation. As summarized in Bishop's book, The Illusion of Public Opinion, between 30 and 40 percent of Americans will offer opinions on legislation they have never heard of.

Comment on this week's Mystery Pollster column at Pollster.com.

What social scientists conclude from these experiments is not that respondents lie but rather that, when pressed for an answer, many will form an opinion on the spot, drawing cues from the words or concepts conveyed by the question.

Of course, most media polls provide respondents with more than just a vague bill title when asking questions about public policy issues. The questions asked about cap-and-trade by CNN and Pew Research, for example, provide brief descriptions of the legislation.

"Under a proposal called 'cap-and-trade,'" CNN told its respondents, "the federal government would limit the amount of greenhouse gases that companies could produce in their factories or power plants. If companies exceeded those limits, they would either pay a fine or pay money to other companies that produced smaller amounts of greenhouse gases."

After hearing that statement, 60 percent of American adults were in favor of the proposal, 37 percent were opposed and 3 percent were unsure.

Pew Research asked a simpler question: "Do you favor or oppose setting limits on carbon dioxide emissions and making companies pay for their emissions, even if it may mean higher energy prices?" Fifty percent of adults favored such limits, 39 percent opposed them and 11 percent were unsure.

Since we know that many respondents will form an opinion on the spot, even if all they have to go on is something as vague as the "the 1975 Public Affairs Act," what are we to make of responses to questions that use possibly unfamiliar terms like "greenhouse gases" and "carbon dioxide emissions?"

Fortunately, the Pew Research Center also asked additional questions to help put these results in context. They found that very few Americans are likely to have a pre-existing opinion of the actual legislation being debated in Congress. Specifically:

-- They found that only 14 percent of adults had heard "a lot" about "a policy being considered by the president and Congress called 'cap-and-trade' that would set limits on carbon dioxide emissions," while 30 reported hearing "a little" and 55 percent reported hearing "nothing at all."

-- On a separate survey, they found that only about a quarter of adults (23 percent) could correctly identify "the 'cap-and-trade' legislation being discussed in Congress" as dealing with "energy and the environment" rather than health care, banking reform or unemployment.

Moreover, Pew found that those paying the most attention to the cap-and-trade debate are very different from those who have not. More Republicans (20 percent) had reported hearing "a lot" about cap-and-trade than Democrats (8 percent), and the small number who are attentive oppose carbon emissions limits as described by Pew by a two-to-one margin (64 percent to 32 percent). Those who have heard nothing favor those limits by a margin of 50 percent to 39 percent.

What should we make of such findings?

I put that question to George Bishop. "'Cap-and-trade' legislation is so obscure and so little-known by the vast majority of Americans," he concluded via e-mail, that questions about it generate the same sort of "pseudo-opinions" as the fictitious 1975 Public Affairs Act. "Reliable and valid measures of public opinion on such a complex policy issue," he writes, "cannot be so simply simulated by merely telling respondents what it's about and then asking them to react to it on the spot. Down that road lie misleading illusions and the manufacturing of public opinion -- a disservice to the Congress, the president and the press that covers them."

My own reaction is a little different. Policy makers have good reason to want to probe the sorts of opinions that the venerable political scientist V.O. Key once termed "latent," those likely to be stirred up should the legislation become law or the focus of a future election campaign. Pollsters can attempt to simulate such hypothetical attitudes in a telephone survey, but the results will be very sensitive to the words they use and, more importantly, to the assumptions they make about the competing arguments voters may eventually hear.

It is possible to anticipate how public opinion on a topic like cap-and-trade may evolve, but proceed with caution. The mistake is to assume that unknown facts will become known, that any particular framing will dominate, or that any one question will magically capture it all.