Most of our staff at the retreat.

Each year for the past three decades, I've savored the joys of New England autumns - crisp and clear days, tasting and even picking fresh apples, enjoying fall foliage, carving pumpkins, cheering on my favorite college football team. Like all seasonal transitions, it's a time of change from past to future. Weatherwise, I'm more fond of looking back at the fleeting summer than I am of anticipating the arrival of winter cold. On the other hand, at this week's annual GRG retreat, we spent less time reflecting back and more time thinking forward. "Change" is a de rigeur word in our evaluation profession; there's even a Theory of Change.

Recently we established a GRG book group, and we've been reading Influencer: The Power to Change Anything by Kerry Patterson, Joseph Grenny, David Maxfield, Ron McMillan, and Al Switzer (McGraw Hill: 2007). While the book has been discussed by folks in a variety of endeavors (e.g., marketing, sales, dieting, program management, education), we have focused on how we can use its principles in our communication regarding evaluations, with both our clients and the end users of their programs, particularly when it comes to recruiting the latter to participate in our evaluation research.

The basic premise of Influencer is that in order to make desired changes, we must first carefully determine which "vital" behaviors are most important to change, and then discover what issues, relating to those behaviors, people believe are in their best interest, and finally appeal to those beliefs. This motivation works better than trying to directly change behavior. Often, reinforcing just one belief can change the one vital behavior, which can then solve a complex problem and create remarkable changes. When asked to change their behavior, people ponder two questions: Is it worth it for me to change? (Motivation), and Can I do it? (Ability). Influencer discusses six strategies for approaching each question through each of three perspectives: Personal, Social, and Structural Influence:  

1. Personal Motivation: Make the Undesirable Desirable (overcome reluctance and resistance)
2. Personal Ability: Surpass Your Limits (learn to master necessary skills for success)
3. Social Motivation: Harness Peer Pressure (enlist the help of others)
4. Social Ability: Find Strength in Numbers (teamwork)
5. Structural Motivation: Design Rewards and Demand Accountability (reward yourself early)
6. Structural Ability: Change the Environment (surround yourself with supportive physical environments)

The book is highly readable, full of examples and case studies, and relevant to anyone interested in making positive change.

Wishing you a wonderful season,

GRG adds yet another cutting-edge statistical technique to its repertoire: Multiple imputation of missing data.

Chances are you've encountered this situation: Your program is collecting data from a set of at least two informants (e.g., students and their parents, professors and their students, families visiting a museum, clinicians and their patients) and you have missing responses from one of the pair. What to do? You don't want to simply exclude the missing pairs from analysis, because then the remaining sample would likely be biased. That is, people who complete surveys probably have some characteristics in common, and these are probably different from the characteristics of people who skip surveys.

Instead, we need a way to estimate missing responses on key variables for purposes of analysis. Multiple imputation is a statistically principled way of creating a set of "best guess" estimates based on the information we do have, including how variables are interrelated within the existing paired data and what variables predict missing survey data. The method also adjusts for the fact that we should be less confident in an estimated or imputed value than in an actual value provided by the participants. Analyses are performed on a set of imputed values, and results are combined.

Extensive studies have shown that multiple imputation is an effective and accurate method for dealing with missing data on key variables, provided there are enough relevant variables in the existing data to allow for good estimates of missing data. It can be used even with as much as half the data missing.

(For an accessible introduction to multiple imputation, click here.)

NSF Revises Merit Review Criteria

To those of you who apply for funds from NSF, please take note that on October 4, NSF revised their Proposal & Award Policies & Procedures Guide (PAPPG), effective starting with proposals submitted this January 14, 2013 (NSF 13-1). A summary of all the changes is here, but prospective PIs may wish to review the merit criteria revisions. Proposals will still be evaluated on their Intellectual Merit and Broader Impacts, but "guidance has been provided to clarify and improve the function of the criteria."