Upcoming Events 

• Thursday, January 12, 2012 - Post Holiday Party.  Palomar Hotel, 117 S 17th Street, Philadelphia, PA. This year we will be collecting gently used clothing for The Career Wardrobe and MenzFit.  [Party details]   
• Thursday, February 9, 2012 - Evening Education.  Westin Hotel, 99 S 17th Street, Philadelphia, PA.  Speaker- Margie Omero, Founder, Momentum Analysis, LLC.  [Margie's bio]   
• Thursday & Friday, April 19 & 20, 2012 - New York & Philadelphia MRA Joint Chapter Conference.  Borgata Hotel, Casino & Spa, Atlantic City, NJ. [Conference details]          

Feature Article

Common Method Bias in Survey Research: An Introduction

G. Walter Wang, Ph.D.

Associate Professor of Marketing, Penn State University

      The survey method is routinely used in market research. Researchers ask respondents such as consumers and business customers for information on their perceptions of product quality, service experience, brand choices, consumption behaviors, and future plans. Data are then analyzed using simple descriptive statistics, or advanced techniques such as linear and nonlinear regression models to detect relationships among variables. Substantive conclusions are then drawn based on these analyses with certain degree of confidence as indicated by statistical significance levels.    

     Yet, market research practitioners seldom realize the very method they use and rely on often produces biases in data that lend statistical conclusions unreliable. Such biases, termed common method variances (CMV), may be due to a variety of factors, including social desirability, the transient mood and emotional state of the respondent at the time of the survey, the context or format of the item scales, the complexity and ambiguity of the questionnaire, and so on. Behavior research has suggested that, on average, approximately one quarter of the observed variance can be attributed to systematic measurement errors such as CMV. CMV can inflate or deflate the value of the measurement or the correlation between variables. It has been found that when two variables are perfectly correlated at 1, method bias can cut the observed correlation in half and deflate the variance explained by 70%. When two variables are completely unrelated, CMV could cause the observed correlation to be greater than zero.  

     CMV is especially profound when the independent (i.e., predictor) variables and dependent (i.e., criterion or outcome) variables are coming from the same respondent with the same data collection method within the same context. These biases, if not properly treated, can give us false information and lead us to erroneous conclusions. Therefore, it is critical to understand the sources of such biases in market survey research and identify proper remedies to control for them.    

The term method here refers to the various measurement characteristics such as the content of the survey questions, scale type, response format, and the general context. CMV can be thought of as random and systematic response biases due to the common characteristics that are present in measuring different variables of interest.    

     There are several noteworthy sources of CMV: common rater, item characteristics, item context, and measurement context. First, biases may be introduced to the observed correlation between two variables when they are rated by the same respondent (i.e., common rater). For example, respondents may try to exhibit an artificial consistency in their responses to different questions (consistency motif), may have the tendency to say no (or yes) to all questions regardless of the question content (acquiescence bias), or may respond to the survey in ways that are consistent to their positive or negative mood or emotional state at that particular moment.   

     Second, item characteristics may have a systematic effect on responses when questions are written in a way to solicit socially desirable or politically correct responses (e.g., "Do you use illegal drugs?" "Are you willing to help the poor and disadvantaged?"), or when questions have implicit or explicit cues as to how to respond to them (e.g., "Since teachers are part of our children's future, do you support a fair contract for teachers?"). People tend not to think too much and will put themselves on autopilot when all questions use the same response format (i.e., 5-point Likert scale). When questions are ambiguous, respondents may interpret the questions in different ways and the resulting answers can be hard to interpret. Scale anchors or response categories such as "never" and "always" may also affect responses since some people are reluctant while others prefer to choose an extreme value as an answer (moderacy bias and extreme response bias).  

     Third, item contexts may affect responses when the first few questions encountered set a positive or negative mood for responding to the rest of the questionnaire, or when a question that appears first affects the way the following questions are understood or answered. Finally, when variables are measured at the same point in time, in the same location, or using the same medium (e.g., paper and pencil, web, telephone), these same measurement contexts may also affect the responses systematically.

     As such, every attempt should be made to understand the possible effect of the method characteristics on survey responses. Depending on the source of the CMV as outlined above, CMV may be remedied through appropriate design procedures. First, effort should be made to separate the measurement of independent and dependent variables in terms of rater, time, and location. Researchers can also consider using different scale formats (Likert scale, semantic differentials, closed or open ended questions), scale end points, and media (paper and pencil vs. web-based vs. face-to-face interviews) to minimize the bias. Second, it is very important to improve the quality of questions and scale items so that they are clear, specific, and concise. Ambiguous or unfamiliar terms should be defined, while double-barrel questions avoided and decomposed. Third, to control item context effect, the researcher can counterbalance the order of questions related to independent and dependent variables so that systematic influence of one question over another is minimized.  

     Not all of these design alternatives can be applied in any research setting and they may not totally eliminate CMV. Thus, statistical remedies should also be considered in conjunction with design procedures. If a particular CMV source, such as social desirability or negative affect, can be identified and measured as a variable, then it could be used in the regression as an independent variable to control for its influence. Alternatively, instead of using zero-order correlations, partial correlations that have partialled out a general factor score through a principal component analysis can be employed for analysis. If the researcher analyzes data with structural equation modeling to account for measurement errors, a single method factor or multiple method factors can be explicitly modeled in the analysis.

Each newsletter will feature a review from a member.  If you recently visited a restaurant, museum, saw a movie or play, etc. and would like to write a review please contact maria.pacenza@gmail.com.  This newsletter features Michael Georgianna's review of the stores and restaurants in the city's Midtown Village.