Newsletter: January 2012Vol 12, Issue 1


A New Year Begins with a Word from AEA President


Happy New Year, colleagues in AEA!


It is with immense honor that I write this column as your AEA President and share my excitement about the year ahead for our incredible organization. Since graduate school days, I can recall being mentored by Bob Covert who made it a habit of introducing me to plenty of others in evaluation by way of work, by way of the annual conference, and by way of his professional network. AEA is family to me and it is that same spirit of collegiality, support, and community that I greet you and hope to demonstrate this year through our professional work and the conference later in the year (22-27 October in Minneapolis, MN).


Our theme, "Evaluation in Complex Ecologies: Relationships, Responsibilities, Relevance" invites us to think about these notions in our practices and methods in more deliberate ways. Concern for relationships obliges us to consider key interests, interactions, variables, and stakeholders amidst dynamic and complex issues at program, policy, and project levels. Attention to responsibilities requires us to consider how our designs, delivery, and dissemination are responsive, ethical, and equitable for instance. We focus on relevance because we consider issues like use, sustainability, scale, and strategy for these same programs, policies, and projects that we evaluate.


Last week, I was "in" Accra, Ghana (through the African Evaluation Association TV webcast) where I saw fascinating conversations about gender equity, evaluation, and development on one particular day. In general, presentations reflected on responsibilities and rights for development evaluators working in diverse, multicultural, multilingual communities. We had a decent showing of AEA members in the audience with documents such as AEA's Guiding Principles referenced on more than one occasion. How exciting to see the network of evaluators growing around the world, how our own organizational presence expands beyond our borders, and more importantly, how our evaluation work appears meaningful in multiple settings.


As a Board, we have a number of engaging projects underway this year as usual and I invite you to become informed and involved as you wish!  One such set of projects around International Listening will help us develop strategies for learning what appropriate roles might be for AEA in the international community. Another emerging set of projects will be around environmental sustainability and the public good, and still others around evaluation policy or the ongoing capability to influence evaluation policies that are critically important to the practice of evaluation. Still, we continue to work on thinking about innovative ways to engage members through internal organizational structures that foster our unique and stimulating association. I look forward to sharing more with you as the year continues.


Lastly, should you have some ideas about presidential strands or ideas to make our conference a tremendously rewarding experience, do not hesitate to contact me!


With best wishes as the year begins to you and yours! 


Rodney Hopson  

AEA President 2012

In This Issue
Walking the Talk - Fiona Cram
AEA 2011 Award Winners
Face of AEA - Meet Amy Germuth
2011 Coffee Break Webinars
TechTalk - Blogging
Book: Qualitative Research
AZENet Event: May 8-9
Evaluation Humor
New Job Postings
Get Involved
About Us
Quick Links
AEA's Values - Walking the Talk with Fiona Cram

Are you familiar with AEA's values statement? What do these values mean to you in your service to AEA and in your own professional work? Each month, we'll be asking a member of the AEA community to contribute her or his own reflections on the association's values.  


AEA's Values Statement

The American Evaluation Association values excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community.


             i.        We value high quality, ethically defensible, culturally responsive evaluation practices that lead to effective and humane organizations and ultimately to the enhancement of the public good.

             ii.        We value high quality, ethically defensible, culturally responsive evaluation practices that contribute to decision-making processes, program improvement, and policy formulation.

            iii.        We value a global and international evaluation community and understanding of evaluation practices.

            iv.        We value the continual development of evaluation professionals and the development of evaluators from under-represented groups.

             v.        We value inclusiveness and diversity, welcoming members at any point in their career, from any context, and representing a range of thought and approaches.

            vi.        We value efficient, effective, responsive, transparent, and socially responsible association operations.


CramKia ora - my greetings to you. I'm Fiona Cram, a Māori (Indigenous) evaluator living and working in Aotearoa, New Zealand. I run Katoa Ltd., a small Kaupapa Māori  (by Māori, for Māori) research and evaluation company. I've been a member of AEA for seven years and involved mainly in the Indigenous TIG. In 2012 Ricardo Millet and I are conference strand co-chairs, working alongside Rodney Hopson, the AEA President.


The values statement speaks to what AEA believes and is committed to; it is the heart of the organization. In my first encounters with AEA I asked myself 'Is this a place for me?' In answering this question I looked to: members' friendliness and willingness to share their expertise, the sorts of evaluations being conducted, and the growing diversity among the membership. I found that AEA strives to practice the values it espouses. The creation of the Indigenous TIG soon after I joined formalized a corner within AEA for the exploration of how the values are reflective of, and embedded within Indigenous evaluation.


My evaluation practice within Māori organisations and communities is often for government agencies, sometimes for philanthropic funders, and now and again for tribal authorities interested in programme efficacy. When I reflect upon the AEA values as they apply within this context I note that:

  • Māori communities continue to be cautious about evaluation. They are concerned about whether evaluators will understand them and how they will be portrayed. Undertaking culturally responsive evaluation within this context begins with building trust relationships and developing mutual understandings.  AEA values express my willingness to undertake this journey.
  • Māori have the right to excellence in evaluation practice and AEA colleagues inform my understanding of what this means. Within this global community we explore how we are the same and how we are different, as well as what we can learn from one another. This challenges me to be more explicit about my practice so that I might reciprocate this sharing.
  • Māori evaluation capacity is growing. When I support evaluators at the beginning of their careers I know that AEA will also value and encourage their participation.
  • Finally, the values are a reminder that my evaluation practice should help facilitate 'humane organizations' and 'public good', and I continue to ponder what these mean for Māori.

AEA values are compatible with Kaupapa Māori evaluation and the development of Māori evaluation capacity. They provide a platform from which we can engage with others without having to renegotiate the fundamental understandings that are espoused.

AEA Recognizes Outstanding Service & Contributions 

The American Evaluation Association honored four individuals and three groups for outstanding work at its Evaluation 2011 conference in Anaheim, CA. Honored this year were recipients in six categories who have been involved with cutting-edge evaluation/research initiatives that have impacted citizens around the world. We spotlight our remaining two below. Others were profiled in earlier issues. Our congratulations to all!     


Idaho Legislature's Office of Performance Evaluations

2011 Alva and Gunnar Myrdal Government Award


IdahoThe Idaho Legislature's Office of Performance Evaluations (OPE) was recognized for work that led to changes in policy and legislation as well as favorable media coverage. OPE's work led to (1) more streamlined efforts and new initiatives - including the creation of a drug czar position - to effectively deal with Idaho's substance abuse issues, (2) substantive organizational changes at the Idaho Department of Health and Welfare, and (3) better ways of managing Idaho's transportation infrastructure assets and saving millions of dollars in highway construction, maintenance, and preservation.  


Laura Leviton, Laura Kettel Khan, Nicola Dawkins

2011 Outstanding Publication Award

The Systematic Screening and Assessment Method: Finding Innovations Worth Evaluating    

Publication Countless programs are launched annually. Which ones merit evaluation for outcomes and how do we know? The Systematic Screening and Assessment Method: Finding Innovations Worth Evaluating (SSA)describes a cost-effective way to assist program funders, practitioners, and researchers in selecting the most promising innovations already in use and then prepares them for more rigorous evaluation. SSA combines existing evaluation methods into a six-step process, which has been adopted or adapted by the Centers for Disease Control and Prevention's National Center for Chronic Disease Prevention and Health Promotion, Division of Nutrition, Physical Activity, and Obesity; Division of Heart Disease and Stroke Prevention; Division of Cancer Prevention and Control; and the National Center for Injury Prevention, Division of Violence Prevention. In addition, it is recommended by a consensus panel of the Institute of Medicine and cited by the General Accountability Office.


"It is an honor to lead an association with the caliber of dedicated professionals like our award-winners," says AEA President Jennifer Greene. "Their work demonstrates the substantial value of evaluation to diverse policy and program arenas in our society and around the globe." 


It's not too early to start thinking of AEA's 2012 awards! Stay tuned for more information on AEA's awards nominations and upcoming deadlines.    


Go to AEA's Awards Page  

Face of AEA - Meet Amy Germuth, Independent Consultant 
AEA's more than 7,000 members worldwide represent a range of backgrounds, specialties and interest areas. Join us as we profile a different member each month via a short Question and Answer exchange. This month's profile spotlights Amy Germuth, an independent consultant whose presentations on improving survey quality are among the most downloaded in AEA's eLibrary.



Name/Affiliation: Amy A. Germuth, Ph.D., President, EvalWorks, LLC
Degrees: B.S. Mathematics, University of North Carolina at Chapel Hill; M.Ed. - Ed. Administration, Tennessee State University; Ph.D. Education Psychology, Evaluation, and Measurement, University of North Carolina at Chapel Hill; Certificate in Survey Methodology, University of North Carolina at Chapel Hill
Years in the Evaluation Field: 12
Joined AEA: 2000
AEA Leadership Includes: Chair, Independent Consulting TIG 2009; Chair, Data Visualization and Reporting TIG 2012; Publications Committee Chair; AEA Guiding Principles Review Task Force member


Why do you belong to AEA?

"I joined AEA to learn more about evaluation, to meet other evaluation-oriented persons, and to establish a professional network. I've met most of the evaluators with whom I collaborate at AEA's annual conferences along with scores of others with whom I hope to collaborate some day!  Every day I get the aea365 post in my email, which provides me with new ideas or tricks, and reminds me of the value of such an organization. Lastly, AEA provides me an opportunity to give back to other evaluators and those in the evaluation community."


Why do you choose to work in the field of evaluation?

"I fell into evaluation while working as a graduate student and realized it meshed with my interests of education and research. However, as opposed to educational research, I found that evaluation provided me an opportunity to focus on program improvement while using many of the same research methodologies. I liked the practical nature of evaluation and working with stakeholders. Evaluation also allowed me to pursue my other passion, which is consulting. I own an evaluation and survey research firm in Durham, NC, conduct evaluations all over the U.S., including Alaska, and mentor other evaluators who are thinking about becoming a consultant."   


What's the most memorable or meaningful evaluation that you have been a part of - and why?

"One project that was very meaningful was an evaluation I conducted with a colleague recently, related to dislocated workers. The evaluation utilized a mixed-methods design and provided some very interesting and useful results. Our clients loved our work and met with us multiple times to try to understand what our findings meant for their program and the persons they serve. To find clients who are equally as passionate about the evaluation as the program being evaluated is a rare treat! This experience also helped me identify ways I can better engage clients to spark their passion for evaluation."


What advice would you give to those new to the field?

"Join AEA! It's that simple. There, you will find other persons with similar interests and much information to share. Having such a supportive network is especially meaningful if you are a solo practitioner or work independently. And everyone I have met at AEA has been welcoming and appreciative of others. It's an especially great place for graduate students to present and get introduced to this profession."


If you know someone who represents The Face of AEA, send recommendations to AEA's Communications Director, Gwen Newman, at [email protected].

2011 Coffee Break Webinars
From Stephanie Evergreen, AEA eLearning Director


Did you know that AEA offered over 40 Coffee Break webinars - free for members - in 2011 and that you have access to the recordings in the webinar eLibrary?  


You'll need your AEA username and password in order to view the recordings, so just in case you don't have them handy, click here to have them sent to you via email.


Here are the titles from 2011, with direct links to each one:

Thank you to all of our 2011 speakers who made it a banner year!  


And what's on for 2012? Take a look at the lineup, and register for any or all, on our Coffee Break Webinars website. Next up? Gail Barrington on starting a consulting practice. Oh yeah. 

 TechTalk - Evaluators are Blogging
From LaMarcus Bolton, AEA Technology Director


Why blog? We asked over 20 evaluation bloggers just that, as well as "What tips do you have for other evaluation bloggers?" and "What are your favorite posts?" This winter, their responses will be highlighted via an ongoing bloggers series on aea365, with each post featuring the details of one evaluation blog. Following the series is a great opportunity to identify possible writers to follow, as well as to gain insight into blogging if you are considering starting your own. You can access the ones that have already gone online via the aea365 archives or the links below. To subscribe to aea365 via email or RSS (What is RSS?), and receive a Tip-a-Day, including the next installments of the bloggers series, go to the aea365 homepage. From here, use the "Take Action" links in the upper right-hand corner. 


We've already had two weeks devoted to the series, one in December and one in January. With two more to go, I'm already struck by the variety of contributions and diversity of voices represented. So far we've had:




And there is still more to come in February and March! If you are writing an evaluation blog, and would like to contribute to the series, but have not yet heard from us, please email me at [email protected] and I'll get you connected to the aea365 coordinator.

Qualitative Research: An Introduction to Methods and Designs

QualitativeAEA members MaryLynn Quartaroli and Frances Riemer are co-editors of a new book published by Jossey-Bass. Qualitative Research: An Introduction to Methods and Designs features 18 chapters, each written by an expert in the field.


From the Publisher's Site:

"The authors-noted scholars and researchers-provide an up-to-date guide to qualitative study design, data collection, analysis, and reporting. Step by step, the authors explain a range of methodologies and methods for conducting qualitative research focusing on how they are applied when conducting an actual study. The book includes methods of data collection, specific approaches to qualitative research, and current issues in the field. Specifically, chapters cover the methods, designs, and analyses related to the methodologies of history, case study, program evaluation, ethnography, autoethnography, narrative, life histories, emancipatory discourses, feminist perspectives, African American inquiry, indigenous studies, and practitioner qualitative research."


From the Editors:

"Interacting with this exceptional group of scholars from so many disciplines and geographic locations to produce a coherent and useful text has been a highlight of the project," notes Quartaroli, who adds that the book is unique from others on qualitative research in three distinct ways:  


1) It includes many methodologies that are frequently overlooked.

2) Besides the editors' introductory materials, each chapter is written by experts in these topics.

3) The intended audience is not just beginning (or practicing) researchers. The primary aim of the book is to offer readable, accessible content for the professional practitioner (including undergraduates who will soon begin their careers) as informed consumers of qualitative research findings.


Adds Riemer: "I loved planning this book. I loved brainstorming possible chapters with my co-editors, thinking, talking about what methodologies represent the current continuum of qualitative research, and then debating about who might best represent those ways of thinking.  And I'm so pleased that I was part of Steve's last project on research methodologies." 


About the Editors:

Stephen D. Lapan served as Director of the C & I Doctoral Program at Northern Arizona University and was honored with the Arizona Association for Gifted and Talented Honor Board Life Achievement Award, NAU's College of Education Distinguished Service Award for Research, and NAU's first campus-wide Teaching Scholar Award.


MaryLynn T. Quartaroli is the Undergraduate Research Coordinator in the Office of the Vice President for Research at Northern Arizona University. She is also evaluating programs funded by the U.S. Department of Education in projects as diverse as the Math Science Partnership and the Carol M. White Physical Education Program. With Lapan, she edited Research Essentials: An Introduction to Designs and Practices, also published by Jossey-Bass.


Frances Julia Riemer serves as Professor in the College of Education and in Women's and Gender Studies at Northern Arizona University. She is the recipient of a Fulbright Scholar Award, a dissertation fellowship from the Spencer Foundation, a post-doctoral fellowship from the National Academy of Education/Spencer Foundation, and an Elva Knight research grant from the International Reading Association.


Go to the Publisher's Site

AZENet Event May 8-9: Building Capacity for Organizational Effectiveness
The Arizona Evaluation Network (AZENet) will hold its annual spring conference May 8-9 in Tempe, Arizona. This year's theme will be Building Capacity for Organizational Effectiveness.

Day 1 will focus on answering the question: What does research show leads to organizational effectiveness in three key areas: Adaptive Capacity/Strategic Learning, Leadership, and Management Practices? Day 2 will focus on decision-making when investing in capacity building, that aligns with research on organizational effectiveness.
Both days run 9 AM-3 PM.

The keynote speaker will be Peter York, a Senior Partner and Chief Research and Learning Officer at TCC Group. As the field learns more about what it takes to help organizations become effective, some long-standing assumptions about what constitutes high-quality capacity building are being challenged. TCC Group (TCC) has designed, implemented, managed and evaluated over 70 capacity building initiatives, organizations and projects over the past 12 years. Through this body of work, TCC has amassed a significant amount of data and research findings which, in aggregate, are shedding light on what works and what does not when it comes to nonprofit organizational capacity building. More information is available at The event is co-sponsored by the Arizona Alliance of Nonprofits, the Community Foundation for Southern Arizona, and the Phoenix Young Nonprofit Professionals Network.

For more information or to register, go to the AZENet web page.
Evaluation Humor - The Million Dollar Question
We recently invited members to contribute their humorous thoughts showing the intersection of evaluators and others ... We received a variety of laugh-provoking entries. Enjoy!  children
Our thanks to Robert Hoke for one of our favorite entries, above, and one I'm sure we can all relate to!  
Have a cartoon to share? Send your suggestions to AEA's newsletter editor, [email protected]. 
New Jobs & RFPs from AEA's Career Center  
What's new this month in the AEA Online Career Center? The following positions have been added recently: 
  • Assistant/Associate Research Scientist at The College Board (Newtown, PA, USA) 
  • Program Analyst, Monitoring and Evaluation at USDA/Foreign Agricultural Service (Washington, DC, USA) 
  • Director of Academic Assessment at Saint Leo University (Saint Leo, FL, USA)   
  • Senior Researcher at The American Institutes for Research (Chicago, IL, USA)
  • Research Analyst at University of Massachusetts Donahue Institute (Hadley, MA, USA)   
  • Impact Evaluation Consultant for Maternal and Child Health Program at Mercy Corps (Dushanbe, TAJIKISTAN) 
  • Research Associate I at Research for Better Schools (Philadelphia, PA, USA)
  • Research Assistant Professor (Health Home Data Analyst) at University of Missouri, Institute of Mental Health (MIMH) (St. Louis, MO, USA)
  • Sr. Monitoring & Evaluation Specialist - Botswana at International Training & Education Center for Health (Botswana, SOUTH AFRICA) 
  • Evaluation and Strategic Learning Specialist at Children's Law Center (Washington, DC, USA)

Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. According to Google analytics, the Career Center received more than 4,000 unique visitors over the last 30 days. Job hunting? The Career Center is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee. You can also sign up to receive notifications of new position postings via email or RSS feed.


Get Involved
About Us
The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.


The American Evaluation Association's mission is to:
  • Improve evaluation practices and methods
  • Increase evaluation use
  • Promote evaluation as a profession and
  • Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275