From Stephanie Shipman, Evaluation Policy Task Force Member

On February 13, 2012, the United States Office of Management and Budget (OMB) issued the President's Budget of the U.S. Government, Fiscal Year 2013 in several volumes, proposing policy initiatives and agency expenditures for the year starting in October. The Analytical Perspectives volume articulates the Administration's continued commitment to program evaluation and demonstrates an increasingly sophisticated approach to evidence-based decision making. This echoes the stance urged in AEA's Evaluation Roadmap and several years of Evaluation Policy Task Force discussions with OMB officials.

The Administration continues to stress the importance of delivering a more effective and efficient government through cultivating a culture of continuous learning and performance management - "the use of performance goals, measurement, and regular data-driven reviews to drive significant performance gains." This approach builds on practices begun in previous years and institutionalized in the GPRA Modernization Act of 2010 (GPRAMA; summary).   

As in previous years, the Administration proposes to re-invigorate federal government evaluation through: supporting agency funding for research and evaluation; building agency capacity (through both interagency networks for sharing expertise and training opportunities); and institutionalizing agency evaluation policies (see discussion of the State Department's new policy in last month's column). In addition, several agencies have adopted a tiered approach to grant-funding to encourage: scaled-up adoption of practices demonstrated to be effective; validation or replication demonstrations of promising practices; and development and testing of innovative ideas.  

The Program Evaluation chapter, newly titled "Program Evaluation and Data Analytics," describes a much more comprehensive "evidence infrastructure" than in the past for using evidence in program management and policy making.

• Evidence-based policy making is defined broadly to include: needs assessments to inform policy design or revision, monitoring performance, identifying promising practices and challenges, targeting program resources to needs, identifying program effects, and gaining insight into how to improve program performance. 
• As before, it strongly encourages "appropriately rigorous evaluations" (i.e., using context-appropriate methods) to determine the impacts of programs on outcomes and compare the effectiveness of alternative policy choices.
• Agencies are encouraged to "embrace a culture where broad statistical data series, performance and other measurement, evaluation, and other data analytics are regularly used and complement one another."
• Data analytics - a recent addition - is described as "the analysis of patterns, relationships, and anomalies in administrative and other data to inform priority-setting, program design, and hypothesis formulation."

To foster use of evidence in decision making, GPRAMA instituted quarterly progress reviews for both government-wide and agency priority goals. Agency leaders are expected to support an evidence infrastructure and sustain a culture of learning, while evaluators play a key role in providing analysis and insight to help inform operational and policy decisions and identify the need for new data and evaluation studies.   

Go to the EPTF Webpage 

Are you familiar with AEA's values statement? What do these values mean to you in your service to AEA and in your own professional work? Each month, we'll be asking a member of the AEA community to contribute her or his own reflections on the association's values.  

AEA's Values Statement

The American Evaluation Association values excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community.

             i.        We value high quality, ethically defensible, culturally responsive evaluation practices that lead to effective and humane organizations and ultimately to the enhancement of the public good.

             ii.        We value high quality, ethically defensible, culturally responsive evaluation practices that contribute to decision-making processes, program improvement, and policy formulation.

            iii.        We value a global and international evaluation community and understanding of evaluation practices.

            iv.        We value the continual development of evaluation professionals and the development of evaluators from under-represented groups.

             v.        We value inclusiveness and diversity, welcoming members at any point in their career, from any context, and representing a range of thought and approaches.

            vi.        We value efficient, effective, responsive, transparent, and socially responsible association operations.

From Victor Kuo, AEA Board Member 2011-2013

This year, the AEA Board of Directors is continuing one of its key initiatives to develop policy on the Association's role internationally. I am pleased to provide an update to our readers on the progress we are making building on the momentum from last year's accomplishments.

1)2012 Project Task Force. This year's Task Force is composed of three board members, a representative from the International and Cross Cultural TIG, two representatives from other international regional evaluation associations, and one additional AEA member. The Task Force is charged with developing policy for the Board's consideration regarding AEA's role internationally.

2)Board Conversations to Refine AEA Policies. This year's Task Force is partnering with a consultant to further synthesize findings from last year's efforts and to facilitate a strategic conversation with the Board of Directors. This work includes seeking feedback from a limited set of external experts on the strategic implications of policy options. The result will be suggested refinements to AEA's policy language and guidance for future activities.

3)AEA 2012 Conference Activities. The Task Force plans to host a Think Tank session this year in Minneapolis to review progress made to date and to engage the membership in a conversation about implications and how members can get involved.  Plans are also underway to engage leaders from other international evaluation associations in dialogue to identify opportunities for mutual collaboration.

Consistent with the spirit of the International Listening Project, we invite AEA members' inquiries and feedback on the Task Force's work throughout the year. To review the Association's current policies, including those addressing international programs, click here. On behalf of the International Listening Project Task Force, we look forward to engaging with you!

Welcome to AEA's newest staff member, Karen Anderson. Karen, who serves as AEA's Diversity Coordinator Intern, will be supporting AEA diversity programs and topical interest groups (TIGs), as well as its Cultural Competency Task Force.  

Currently working as a Continuous Quality Improvement and Compliance Associate at Families First in Atlanta, Karen received her undergraduate degree in Psychology from Clemson University in 2008 and her Masters in Social Work from Clark Atlanta University in 2010.  

"I fell into evaluation like many people, by happenstance. I had a strong interest in research, and I wanted to develop parenting programs, but I needed to learn what made them tick. After I worked on a few evaluation projects in grad school I was hooked!" says Karen. "The learning opportunities that come along with evaluation seem to be endless, which is another attractive part of the profession for me. In evaluation, you have the ability to take on countless roles and there is always a new setting to apply your evaluation skillset."

An active member of AEA, Karen is coming up on her last term as Social Work TIG program co-chair, and she was also a Graduate Education Diversity Intern (GEDI) with the Evolution cohort in 2009-2010. In addition, she has volunteered evaluation assistance to a lesbian, gay, bisexual, and transgender organization in Atlanta, GA for the past 3 years and has had an array of evaluation experiences in nonprofit and for-profit settings.  

"My Graduate Education Internship Program (GEDI) experience with AEA provided a very firm foundation for me on my journey to become a culturally competent evaluator," Karen notes. "As a GEDI, I was inundated with literature from Stafford, Hood, Hopson, Symonette, the list goes on, and I was exposed to many opportunities to have discussions about culture and evaluation. If you are also on the cultural competence journey, AEA's public Statement on Cultural Competence in Evaluation is an amazing resource for pointing you in the right direction," says Karen.

Stay tuned as Karen is developing an online diversity portal that will house AEA resources focused on diversity with an anticipated launch date of this fall. She also will be writing a diversity column for upcoming newsletters and will seek input from AEA members interested in sharing experiences highlighting diversity in evaluation.

"Diversity, the word in itself is so broad, and in a good way. This will help me to stay outside of the box when I am supporting various individuals and groups in the association," says Karen. "I think it's great that AEA has programs, TIGS, policies and statements that embrace diversity and inclusion. From my experience, the AEA culture embraces all evaluators young and young at heart, across the U.S. and around the world so everyone feels like they have a home."  

If you would like to learn more about the current AEA diversity initiatives, please email karen@eval.org.

Join us in welcoming Tessie Catsambas as AEA's official representative to the International Organisation for Cooperation in Evaluation (IOCE). IOCE serves as a hub or support mechanism for evaluation associations, societies and networks around the world in order to promote the strengthening of the worldwide community of evaluation professionals.

Anastasia (Tessie) Tzavaras Catsambas is President of EnCompass LLC and brings 25 years of experience in planning, evaluation and management of international programs and activities. She is an innovator and practitioner in appreciative evaluation methods. Starting in 1998, she experimented with the use of appreciative inquiry in an evaluation of healthcare quality assurance activities in Chile. This became the first published work in appreciative evaluation in the Journal for International Health Care Quality, Volume 14, Supplement I, December 2002. She has continued to refine appreciative evaluation methods through her work with many clients in diverse settings including the World Bank, the United Nations, The Aspen Institute, the U.S. National Academies of Science, USAID, CARE International, the International Women's Media Foundation, and others. Tessie holds a Bachelor's degree in Economics and Political Science from the College of Wooster in Ohio, a Master's degree in Public Policy from Harvard University, and trained with the late W. Edwards Deming in Quality Management. She is fluent in French and Greek, and speaks Spanish.

Tessie also has a rich history within AEA. She served as a co-Chair of the International and Cross-Cultural Evaluation Topical Interest Group (ICCE) for four years and, for two years, served as the ICCE representative to the IOCE. She has served on AEA's Finance Committee since 2010, was appointed to the IOCE Board in 2011 and is part of the Executive Committee serving as Secretary. Other volunteer work has included supporting a partnership between AEA and UN Women to build gender-responsive evaluation skills at the AEA and globally. She has been a presenter at AEA's annual conference since 2002, has coauthored two chapters on appreciative evaluation methods (Preskill & Coghlan, New Directions for Evaluation #100, 2003), and co-authored with Hallie Preskill a book entitled, Reframing Evaluation Through Appreciative Inquiry (SAGE Publications, 2006).

"By helping to professionalize evaluation around the world, IOCE serves the interests of AEA's members. Many other organizations have a stake in the international scene - the International Development Evaluation Association (IDEAS), the United Nations Evaluation Group (UNEG), the Network of Networks on Impact Evaluation (NONIE), other regional evaluation associations, multi-lateral and bilateral organizations, and even other entities such as MIT's Poverty Action Lab. In this crowded scene, the IOCE serves as a neutral meeting ground. AEA needs to be at the IOCE table reinforcing its commitment to continue to build the evaluation profession, showing respect for other countries and cultures, providing opportunities for learning for our members, forging strategic partnerships across boundaries, and ensuring that we, the AEA, have a voice promoting excellence in evaluation."

Go to the IOCE website  

Name, Affiliation: Matt Keene, United States Environmental Protection Agency; Environmental Evaluators Network
Degrees: B.S. Biology (Virginia Polytechnic Institute and State University); Masters in Environmental Management (Duke University)
Years in the Evaluation Field: ~ 10
Joined AEA: 2010
AEA Leadership Includes: Coffee Break Webinar; aea365; Environmental Program Evaluation TIG; 2012 AEA Conference Task Force; 2012 AEA/Environmental Evaluators Network Forum Unconference Coordinator
 
Why do you belong to AEA?  

"If I'm asked to recommend a professional network worth joining, AEA is the first I mention every time. Predictably, that (one-sided) conversation goes like this: "You must check out AEA! It's simply the best value professional network I've come across. Daily blogs, weekly webinars, monthly journals, heaps of professional development opportunities, THE hub of the global evaluation network, monstrously powerful annual conference, fantastic leadership and amazing management!"

Why do you choose to work in the field of evaluation?

"In undergrad I heard a lot about the rainforest. The notion of saving it made sense to me. In 1999 I walked out of Corcovado National Park (a magnificent protected tract of Costa Rican tropical wet rainforest) to collect a few carambola (delicious but not native) from a nearby farm. I took the same path to walk back into the park. I passed a few fellas with guns and bags full of game. I came upon a square kilometer or two of freshly scorched hillside where old growth virgin forest had stood a few days earlier. I looked out over the now unobstructed view of the Pacific and watched a fishing vessel close a purse seine around its catch. I wondered how well parks worked."

What's the most memorable or meaningful evaluation that you have been a part of - and why?

"The evaluation of the Oregon Paint Stewardship Pilot Program stands out. To navigate some curmudgeonly circumstances, we headed into territory still sparsely inhabited by evaluators, for instance, integrating evaluation into program design, systems thinking, participatory data collection and using art to communicate program and policy space (fuzzy logic models). The purpose of the evaluation was to help other states to learn from Oregon's experience so they could then design and run their own state-wide paint management programs. Today, the evaluation is hard at work all around the U.S."

What advice would you give to those new to the field?

"Understand why evaluation is relevant to this quote: 'Where is the life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?' by T.S. Eliot."

"Evaluation needs adults and children...but a lot more children."

"Because, today, the world drives the evolution of this discipline, evaluation must always be present at the creative margins. It is up to you to make certain that we can comfortably occupy that edgy space."

If you know someone who represents The Face of AEA, send recommendations to AEA's Communications Director, Gwen Newman, at gwen@eval.org.

From Stephanie Evergreen, AEA's eLearning Initiatives Director  

In this column, I interview Victoria Flood, AEA member and frequent webinar attendee, on how to stay focused during a webinar.

Stephanie (S): It can be so hard to block out distractions during a webinar - what with colleagues, email, and so on. How do you keep distractions away?

Victoria (V): I sit in a cubicle, and distractions definitely abound! I try keeping distractions to a minimum by, first, letting my colleagues know I will be participating in a webinar. When the webinar starts, I close all the programs on my computer. This helps minimize my distractions and ensure my machine runs smoothly to receive the webinar. Finally, I clear a little space around my computer so that I'm not tempted to read something nearby.

S: You just finished taking a 6-hour eStudy course on correlation and regression. That course was broken into 90-minute chunks. How did you keep yourself focused for 90 minutes at a time?

V: Fortunately, with interesting topics, staying focused is not too challenging! I do a little prep work ahead of time by reviewing the handout and then having it accessible when the webinar starts. Another way I stay focused is to challenge myself to think of a question (or two) related to what is being presented. Whether or not I type it in, just thinking of questions keeps me engaged.

S:What do you do after the webinar is over to retain the information you learned?

V: When I can, I share something of what was presented with a colleague of mine. This really helps me retain what I learned and begin to think about how I can use it in my current projects. And, when I find I can't explain something as well as I'd like to, then I know I need to review the presentation slides or check out the recorded version when it becomes available!  

S: Any other tips for making the most of the AEA webinar experience?

V: I suggest participating in as many webinars as possible. Especially with the Coffee Break webinars, even if the topic is something I am not sure is relevant to the work I do, I still sign up. I have been introduced to so many new ideas, programs, tools, and tips - and, there is always the chance afterwards to make suggestions for other AEA webinar topics!   

---------------

What's ahead? Registration is now open for Tina Christie on Evaluation Theory and Michael Quinn Patton with a two-part series on Developmental Evaluation. Head over for more details.

How do you communicate with your clients? AEA likes to spotlight samples of great client and stakeholder communications. Here we connect with Cassandra O'Neill  with Wholonomy Consulting, shown here with Sarah Griffiths, both senior partners.   

Wholonomy Consulting is an Arizona-based consulting firm dedicated to building capacity for effective engagement and decision making. "One of the ways we do this is through training on coaching, group facilitation and high engagement presenting. As a result of our facilitation and training, we have developed materials to support the learning of participants that we share with others interested in increasing engagement."

How do you connect with clients?

Our newsletter is something that we use to communicate about upcoming events and trainings open for registration. For example, our last newsletter included information about three events we helped design and facilitate, including the Arizona Evaluation Network's Spring Conference on May 8-9, which is focusing on Building Capacity for Effective Organizations. As part of this event, we are inviting people to host small group discussions on how they are practicing strategic learning, a core competency for effective organizations. 

Our Evolutionary Sustainability blog is used to share photos from events, tips and information, and reflections on our work with groups. The most recent post was about using Gallery Walks - an engagement strategy that can be used to present information and share information generated during small group discussions.  

We also have two publications on ideaencore, an online resource for organizations in the social sector, one free of charge. The Bubble Sheet - A Flexible Action Planning Tool is a culmination of our collaborative work with early childhood professionals to develop a thinking and action planning tool -- that aligns with how the brain works.  

The Six Secrets of High Engagement Presentations grew out of our trainings on this same subject. We've attended a lot of workshops on how to use engagement strategies other than lecture, and we wanted to help share what we were learning from our experiments. We have tried to make it easy for people to try one or two things and then try a few more. We have recently gotten this translated into Spanish and are in the process of putting that version up for sale. We are also in the process of getting this turned into something that can be read by a Kindle.   

Fascinated? If so, you can learn more about how to actively engage participants through an upcoming AEA Coffee Break webinar on Thursday, May 2. Participation is free but registration is required. You can preregister here.

To share samples of the ways that you interact with your audiences, email AEA's Communications Director, Gwen Newman, at gwen@eval.org. We'd love to share the ways you communicate via this column as well as AEA's online eLibrary. Thanks!

AEA member Cindy Banyai is author of Community Capacity and Development: New Approaches to Governance and Evaluation, a new book published by Lambert Academic Publishing.

From the Publisher's Site:

"Broadening the view of development beyond economics is necessary to reduce poverty. Alternative development with a special focus on rural areas and communities is proffered as a more diverse way to address pressing international development needs. Community capacity provides a framework for understanding the comprehensive nature of alternative development. This book unpacks the concept of community capacity and its important relationship to governance, elaborating on the understanding of community to include the local government, organizations and all stakeholders. Governance issues such as decentralization, localization, democracy and participatory practices are advocated to build community capacity and affect development. Tools to improve governance, including the logical framework, policy management cycle and evaluation are explored. The new concepts of community-driven economics and community leadership are introduced to address alternative development and community capacity building. The new A-A-A framework of community capacity develops through a post-modernist, post-positivist, qualitative look at case studies of rural revitalization in Japan and the Philippines."

From the Author:

"One thing that sets this book apart from others in the field," says Banyai, "is that it was written by an American fully emerged in the Asian context. This is meaningful because much cross-cultural work is produced from the inside looking out or the outside looking in. I had been living and working in Asia for nearly six years when this research was conducted, providing a different perspective than someone who had just traveled there to conduct research. Furthermore, this research was conducted in conjunction with Japanese practitioners and scholars who come from a different point of view than their American counterparts when it comes to understanding and applying evaluation. I hope that it may be useful for others working in an Asian context." She adds: "I really enjoyed travelling to the rural villages in Japan to hear their proud stories. I also enjoyed my time becoming immersed in village life in the Philippines. I became very close with many members of the community there and brought back many interesting stories that could not be included in the book, like learning to cha-cha from Ilocos Norte's former governor Michael Marcos-Keon at the town's fiesta."

About the Author:

Cindy Banyai received her PhD from Ritsumeikan Asia Pacific University in 2010. Her research interests are community development, participatory governance and innovative data gathering techniques. She worked as an international development consultant from 2006-2009 and launched the Refocus Institute, a participatory evaluation consultancy, in 2010.

Facebook Timeline Gurus: Do you use Facebook? Did you know it is changing its format so that there is an extended timeline, and that you can add to that timeline milestones and significant historical dates and events? We're looking for a few good volunteers to join a working group to assist with developing and maintaining AEA's Facebook timeline. The task would involve researching and adding milestones from AEA's history to the timeline, such as the dates new TIGs were formed, new affiliates recognized, major programs started, our conference history, etc. This would be a six-month project, beginning in April of 2012 through September of 2012 so that we have a fleshed-out timeline in time for sharing more broadly at the annual conference in October. The project would involve attending a 1-hour phone-based planning meeting and then additional half-hour project update meetings once per month, working collaboratively with a small group via email, and then approximately 3 hours per month of project time, at your convenience, spent identifying and uploading listings to the AEA Facebook page. If you are interested in working on this team, connecting with other AEA volunteers and leaders and creating a lasting record for the association and field, please send an email to kate@eval.org by Monday, April 9, with your name, a brief statement about your interest, and anything that you can share about your use of Facebook.

John Araujo * Kas Aruskevich * Lisanne Brown * Mary Butler * Laurie Clayton * Mike Coplen * Mary Davis * Mark Domingue * Dorothy Ettling * Don Glass * Phoebe Leung * Laura Linnan * T. Julio McLaughlin * Leah Neubauer * Cory Newhouse * Cynthia Phillips * Sharon Rallis * Anne Rusher * Darlene Russ-Eft * Sue Ann Savas * Nick Smith * Rubayi Strivastava * Donald Yarbrough 

• Director of Evaluation and Research at HARC Inc (Palm Desert, CA, USA) 
• Director of Program Evaluation at City Harvest Inc. (New York, NY, USA) 
• Data Manager, Office of Institutional Effectiveness at Webster University (St. Louis, MO, USA)   
• Educational Researcher (Research Specialist or Research Associate) at SEDL (Atlanta, GA, USA)
• Research & Metrics Assistant at Population Services International (Washington, DC, USA)   
• Director, Technical at FHI 360 (Washington, DC, USA) 
• Sr. Research Scientist at CNA Analysis and Solutions (Alexandria, VA, USA)
• Part Time Field Researcher at Karen Peterman Consulting Co. (Hoboken, NJ, USA)
• Diagnostic & Research Services Consultant at Pearson (USA) 
• Research Associate, Malaria Control Team at Clinton Health Access Initiative (Boston, MA, USA)

Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. According to Google analytics, the Career Center received approximately 2,900 unique visitors over the last 30 days. Job hunting? The Career Center is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee. You can also sign up to receive notifications of new position postings via email or RSS feed.

• Register for an eStudy Web Workshop 
• Complete a Volunteer Profile and Review Volunteer Opportunities 
• Download any of the 500+ Materials Available in AEA's eLibrary 
• Sign up for one of the Free (to members) Coffee-Break Webinars 
• Subscribe to AEA Journals Table of Contents Email Alerts 
• Subscribe to aea365 Tip-a-Day email alerts 
• Subscribe to AEA Career Center New Job Postings Email Alerts 
• Sign up for AEA's Electronic Discussion List EVALTALK 
• Sign up for AEA's Evaluation Policy Initiative Discussion List 
• Sign in to the AEA Website and Join the Thought Leaders Discussion Forum 
• Join the AEA LinkedIn Group 
• Submit your Company Profile for a Free Listing in the AEA Find an Evaluator Directory 
• Follow AEA's Evaluation Headlines and Resources Twitter Feed 
• Read AEA's Cultural Competence Statement 
• Visit AEA's Newsletter Archive for past articles of interest 

• Improve evaluation practices and methods
• Increase evaluation use
• Promote evaluation as a profession and
• Support the contribution of evaluation to the generation of theory and knowledge about effective human action.