STORY OF THE WEEK: How are the Democrats doing? Recent polls say...a lot of things

In the span of a few days last week, a handful of prominent polling firms released national numbers on the generic congressional ballot, and each pointed to a different conclusion about the Democrats' trajectory. Newsweek (final calling on 8/30) showed the Democrats up five points (48% to 43%). A few days later (10/3), a trio of polls suggested otherwise. Gallup showed the Democrats down by a whopping thirteen points (53% to 40%); Rasmussen had them down by just three points (45% to 42%); and ABC/Washington Post reported the Democrats behind by six points (49% to 43%). Zogby then showed the race tied (43% to 43%). What is going on?

As pollsters and campaign consultants, we expect a certain amount of variation between polls. Considering the number of factors that affect polling results---from things we control, such as the sample composition and campaign strategy, to the things that we cannot, such as statistical error---it becomes clear why very consistent polling numbers are the exception, not the rule. Understandably, this can be cold comfort for those who are trying to answer the question, "How are the Democrats doing?" with recent polling data.

To arrive at an answer, we thought it would be helpful to first examine two of the factors that may contribute to the variance we are seeing. Both are also easy to identify.  

The audience surveyed: Survey audiences typically come in three distinct flavors: adults, registered voters, and "likely" voters. For a question like the generic congressional ballot, we typically want to see responses only from the subset of Americans who are likely to show up on Election Day---about one-third of the voting-age population in non-presidential (midterm) elections. However, some polls report generic congressional ballot numbers among all registered voters (about two-thirds of the voting age population) or all voting-age adults.

Why does this matter? It turns out that political preferences become more Democratic as you move out from registered and likely voters (D+1 in Pollster.com's trend estimate) to all voting-age adults (D+6). Consequently, polls that do not drill down to the subset of likely registered voters are likely to overestimate Democratic performance. For example, Gallup shows a 3-point advantage for Republicans among all registered voters, but when they isolate the subset of likely voters---pollster standards vary, but this is usually based on previous vote history and/or questions assessing respondents' intent to vote---the GOP advantage climbs to 13-points.

Notably, this bias is even more pronounced in midterm election years, where turnout is disproportionately lower among key Democratic groups, such as young adults and minorities.

How the audience was reached: There are nearly as many ways to reach survey respondents as there are pollsters, and each reflects a balance between competing priorities (e.g., cost, timeframe, audience, etc.). Nevertheless, the ways in which pollsters reachvoters can affect the numbers. In short, people who answer a survey from a human caller on a landline phone are different from those who respond on a cell phone, or on the internet, or to an automated script.

·        Phone v. Internet:

In political polling, the preferred mode of reaching voters is by telephone (landline and cell). There are a number of reasons for this, but they all boil down to one thing: in almost every case, telephone surveys are still the only cost-effective way for pollsters to reach a random, representative sample of likely voters. Among internet polls that report the generic congressional ballot (e.g., PSB and Zogby), none meets the core threshold of having a truly random, representative sample of likely voters.

For our purposes here, the question is whether telephone polls produce different ballot numbers than internet polls. They do. The Pollster.com trend estimate shows Republicans with a 6-point lead (48% to 42%) in telephone polls and a 2-point lead (45% to 43%) in internet polls. The Democratic-lean of internet polls is even more evident when looking at the seventeen months between May of 2009 and September of 2010. In this span, the Democrats led the GOP each month in internet polls but for just three months in telephone polls.

There are also differences between telephone polls that only reach people on landlines and those that also include cell phones. In short, cell-only respondents are younger, more likely to be African American or Hispanic---and more Democratic. A Pew Research study from September of 2008 found that landline-only surveys underestimated Barack Obama's performance against John McCain by about two points.

·        Human Callers v. Interactive Voice Response (IVR, automated, or "robo" polls):

Political pollsters are more divided in their preference between live callers and automated methods. The top media polls (such as ABC/Washington Post and NBC/Wall Street Journal) and most campaign pollsters prefer using live callers, primarily because there are more methodological safeguards. But several automated polling firms (such as Rasmussen and SurveyUSA) have risen to prominence in recent years, and pollsters are still grappling with the implications of their emergence (see Nate Silver's analysis on this issue here).

Regarding the generic congressional ballot, until recently robo polls exhibited a clear Republican lean. In the seventeen months between May of 2009 and September of this year, robo polls showed the GOP with consistent and solid single-digit leads. Live polls, on the other hand, showed Democrats leading (or effectively tied) as late as July of this year. This stands in sharp relief to the current Pollster.com trend estimate, which shows nearly identical 7-point Republican advantages on live polls and robo polls.

---

Ultimately, for political questions such as the generic congressional ballot, we place the most faith in telephone polls that include cell-only respondents and use human callers to perform the dialing. There are several exceptions to this rule (e.g., we are cautious about polls that do not adjust for large swings in party identification), but we have found that these polls are the most reliable indicators of the Democrats' position heading into November.

Based on this, we conclude that the recent Democratic "bounce" has peaked, at least for the moment. This is based on polls last week from CBS (R+8 on the generic congressional ballot), ABC/Washington Post (R+6), and Democracy Corps (R+6). As we noted last week, however, Democrats still have plenty of time, and flush coffers, to draw contrasts with their opponents, close the gap in enthusiasm between the parties, and ultimately chip away at the Republicans ballot advantage. 

OTHER NEWS FROM THE POLLING AND POLITICAL WORLD

Not 1994: A new Washington Post / Kaiser poll finds that nearly 6-in-10 say they want their congressional reps to fight for more government spending in their districts to spur job creation (39% say cut spending even if it means fewer jobs). This is in contrast to the same time in 1994, when 53% said they wanted to cut spending.

The GOP "Brand": Republican congressional leaders reached a record low in job approval (24% approve) in a new National Journal / Pew Research poll. Dem leaders are at 30% approval.

Ad watch: Check out this great ad from President Obama on behalf of Cedric Richmond, who is challenging Republican Rep. Joseph Cao in Louisiana.

Messaging: Democracy Corps released results and analysis from new polling in their "October Surprise" project. The strongest messages for Democrats revolve around fighting for the middle class, American jobs, Social Security, and Medicare. Check it out here.

Tea Party: 90% of Tea Partiers, and 66% of Republicans, say the views of the Tea Party are "not too extreme" in the new CBS poll.

Politics as football: Appropriate metaphor, and a great ad on behalf of Russ Feingold. Too bad the NFL took issue with the use of game footage.

Campaign spending: The Democratic Governors Association hauled in $10 million during the third quarter of the year, setting a new fundraising record. However, "outside" GOP-leaning groups currently hold a sizeable 7:1 cash advantage. (h/t PoliticalWire)

Senate odds: Nate Silver puts the GOP's odds of taking over the Senate at 24%, up from 22% last week.

Jobs: Although the unemployment rate held this month at 9.6%, it is worth noting that the private sector added 64,000 jobs last month, the ninth consecutive month of gains in this area. 

If you have free time: MSNBC offers up their Top 10 list for political ads from the past week.

Midterms: Charlie Cook predicts that the Republicans have a better chance than not of taking back the majority in Congress (h/t C. Cillizza).

Seriously? Nevada GOP Senate candidate Sharron Angle fears Sharia law may infiltrate U.S. cities, saying "there is something fundamentally wrong with allowing a foreign system of law to even take hold in any municipality or government situation in our United States." 

PUBLIC POLLING

 Follow us on Twitter: AnzaloneLiszt   Like us on Facebook: Anzalone Liszt Research