Newsletter: November 2013

Are you familiar with AEA's values statement? What do these values mean to you in your service to AEA and in your own professional work? Each month, we'll be asking a member of the AEA community to contribute her or his own reflections on the association's values.  

AEA's Values Statement

The American Evaluation Association values excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community.

             i.  We value high quality, ethically defensible, culturally responsive evaluation practices that lead to effective and humane organizations and ultimately to the enhancement of the public good.

             ii. We value high quality, ethically defensible, culturally responsive evaluation practices that contribute to decision-making processes, program improvement, and policy formulation.

            iii. We value a global and international evaluation community and understanding of evaluation practices.

            iv. We value the continual development of evaluation professionals and the development of evaluators from under-represented groups.

             v. We value inclusiveness and diversity, welcoming members at any point in their career, from any context, and representing a range of thought and approaches.

            vi. We value efficient, effective, responsive, transparent, and socially responsible association operations.

AEA's more than 7,800 members worldwide represent a range of backgrounds, specialties, and interest areas. Join us as we profile a different member each month via a short question-and-answer exchange. This month's profile spotlights Ginger Fitzhugh.

 From Cheryl Oros, Consultant to the Evaluation Policy Task Force (EPTF)

Recent reports offer lessons about evaluation policy development and implementation in different arenas. At the federal level, 2010 legislation (Government Performance and Results Modernization Act (GPRAMA)) updated how federal agencies assess their performance. Agencies were required to create chief operating officers (COO) and performance improvement officers (PIO). The Partnership for Public Service, in conjunction with Grant Thornton, examines subsequent changes in "Taking Measure: Moving from Process to Practice in Performance Management," September 2013.  

Based on feedback from agency leaders, the report indicates that enthusiasm for performance assessment has risen and quarterly discussions are happening, but data are not being used at all levels for decision making (leaving budgets and performance still disconnected). Progress toward a performance culture varies widely across agencies, but with relatively little emphasis on evaluation, which can provide in-depth understanding of program performance (compared to performance metrics). Outreach and communication with Congress on performance progress is nearly nonexistent. OPM has identified skills to deal with performance evaluation and measurement, which most agencies lack (particularly in program staff). The report recommends better coordination between those who lead evaluations and those who set goals and are responsible for performance measurement, as COOs and PIOs are not well versed in evaluation. 

At the state level, the Government Finance Officers Association recommended that states develop and use performance measures and measure program outcomes, efficiency, and effectiveness. These should be used for continued improvement as an important component of long-term strategic planning and decision making, which, in turn, should be linked to governmental budgeting ("Eye on the Prize: States Looking at Goals, Outcomes for Budget Decisions, Council of State Governments"). 

The key concept is that programs that can prove their effectiveness with data will be more likely to obtain the funds they seek. Most states claim to use outcome-based budget decision making, and a number of cities have also developed such systems. Every state measures outcomes in at least some agencies, but fewer have a statewide strategic plan to measure outcomes across all agencies. Even fewer relate those outcomes to the budget. For example, both Maryland and Baltimore have developed such systems ("Baltimore's Outcome Budgeting Approach", John Kamensky). Although Baltimore considers evidence when selecting programs for funding, it does not yet have an evaluation system in place to assess the performance of those programs once in operation. 

Internationally, the International Organisation for Cooperation in Evaluation, et al, recently released a report, Voluntary Organizations for Professional Evaluation (VOPEs): Learning from Africa, Americas, Asia, Australasia, Europe, and Middle East (Volume 2 of "Evaluation and Civil Society: Stakeholder Perspectives on National Evaluation Capacity Development"). The report provides information on the approximately 100 national evaluation associations worldwide (with a total membership of about 34,000 evaluators, of which AEA comprises 23 percent). Of particular interest is information on the role VOPEs have played in national evaluation capacity development. Many examples show VOPEs having significant influence on their governments during formulation of high-level evaluation-related policies and as national and provincial systems were established and implemented.  

Please share observations about evaluation policy setting in your federal agency, state, or city (or country) at the EPTF Discussion List or [email protected].

From Zachary Grays, AEA Headquarters

The Minority Serving Institution Fellowship represents one of the many pillars of diversity here at AEA. This year, AEA was proud to grant fellowship to Edilberto Raynes, Denise Gaither-Hardy, Tamara Bertrand, Ana Pinilla, and Andrea Guajardo. Our newest fellows joined us for their first meetings of the cohort during Evaluation 2013. As our participants buckle in for their academic year at their respective institutions, they are also setting out on a year-long journey of professional development and academic enlightenment with many other aspirations in tow. I took a moment to chat with Dr. JoAnn Yuen, a former MSI fellow, to talk about her journey as a fellow, what inspired her to participate, the opportunities the fellowship presented, and why diversity is important to her and should be to the members of AEA. 

Dr. Yuen, a native Hawaiian, greeted me with a warm "aloha" upon my initial contact with her. An associate professor at the University of Hawaii and associate director for the Center of Disability Studies (CDS), Dr. Yuen was encouraged by her mentor and colleague, Morris Lai, to become a part of AEA and the MSI cohort. Dr. Yuen participated in the second MSI cohort during the 2006-2007 academic years. She wanted to join an organization with strong local affiliations in Hawaii while also extending her professional reach and visibility. "As a qualitative researcher, many of the practitioners that shaped my theoretical understanding of the methodology were established and active members of AEA. It was exciting to rub elbows and listen to these individuals in person," she said. 

Though Dr. Yuen (pictured, right) had many opportunities and experiences to share about being a MSI fellow, she highlighted the opportunity to work with Quality Education for Minorities (QEM) during her cohort. "QEM executive director, Dr. Shirley McBay, was a wonderful mentor and has engaged me over the years to collaborate with QEM on grants," she said. "QEM invited me to a training with the NSF program officer. He was in charge of an RFP involving individuals with disabilities and STEM in postsecondary education. Amazing opportunity, though the grant was not funded by NSF. What we produced formed the catalyst for two successful grant submissions."

Dr. Yuen painted an absolutely inspiring portrait of what the MSI Fellowship has done for her and, of course, why the program is important to her, AEA, and why it should be to you as a fellow evaluator. "In the nation, at my institution, and in the field of disability studies, there are concerted efforts to be more diverse, and being native Hawaiian, I have a responsibility to promote this effort," she said. "I have a responsibility to model to other native Hawaiians what it means to be a leader, researcher, a professional who can thrive in academe, and be considered a valued community member. I have a responsibility to mentor, as I have been, to students and faculty." 

Her unbridled passion roars in this empowered statement on diversity that drives home the purpose of this fellowship, providing evaluation resources to those who don't have direct access to the materials at their home institutions. Not surprisingly, participants take this opportunity to provide a voice to the unheard masses in a cross-cultural platform and through the research they conduct. "I feel lucky and grateful to have been included and supported. There continue to be pressures and major efforts (revisions to Sections 501 and 503 of the Rehabilitation Act) throughout the federal government to create diversity at all levels of institutions (University of Hawaii's efforts to increase the employment of Native Hawaiians) and organizations. This is a timeline of my participation. AEA has been ahead of the curve and should continue its commitment to and support of diversity," she said.

Highly regarded and successful prior to participation, Dr. Yuen utilized this opportunity to not only develop herself professionally, but to also provide diverse representation in the evaluation community on her professional projects and products. Today, Dr. Yuen is an advocate for center-wide issues impacting the Center for Disability Studies and CoE and has recently completed a college-wide evaluation on the impact of the Hawaii Department of Education (HDoE), Data Governance Policies (DGO) on student and faculty research. 

"A change in policies within the DGO has created major delays, on average 10 months, in research approvals," she said. "Students across the university are changing research topics and adding one year to their education. Research faculty are in jeopardy of losing extramural dollars because they can't access the data generated through classroom research. Schools will lose thousands of dollars provided by research-related contracts." 

Dr. Yuen will present this report to the dean and share it with the Hawaii DoE with hopes of improving the process. Busier than ever, it is clear that Dr. Yuen's pluck has not wavered since before and during her cohort. It is truly a pleasure to call Dr. Yuen an MSI graduate. After all, as she told me upon outreach, "[MSI] gave me wings." 

Learn more about the MSI Fellowship. 

From Alexa Schlosser, AEA Headquarters 

Our Coffee Break Webinars are short, 20-minute presentations of tools or tips we think evaluators will find helpful in their work lives. As November's webinars come to a close, let's take a look at what's in the pipeline for December:

CBD165: Practical Strategies for Managing Counterproductive Behavior in Small Groups - Robert Kahle
Thursday, Dec. 5, 2013

2-2:20 p.m. ET

Bob Kahle, author of "Dominators, Cynics and Wallflowers: Practical Strategies for Moderating Meaningful Focus Groups" (Paramount, 2007), will describe methods of managing dominant and cynical behavior in focus groups, planning sessions, and meetings of all types. Tips on how to quickly recognize and prevent bad behavior are presented as well as a conceptual tool, the Continuum of Correction, to gently guide small groups back to productivity and inclusiveness. 

CBD166: Pocket Chart Voting - Engaging vulnerable voices in program evaluation - Kerry Zaleski

Thursday, Dec. 12, 2013

2-2:20 p.m. ET

Many people interested in gathering data for program design or evaluation purposes find that their toolkit is insufficient in gathering accurate, reliable, and relevant data in order to give voice to persons living in situations of vulnerability, powerlessness, or marginalization. This coffee break session will provide an example of a participatory method known as "pocket chart voting" that can be used for a variety of purposes, including: needs assessments, program planning, monitoring, and impact evaluation. The method will demonstrate an effective process to confidentially engage vulnerable voices and affirm self-empowerment throughout a decision making process. The tool is adaptable to fit specific contexts and is a good demonstration of a simple, locally-resourced data collection method.

CBD167: Gender-Based Violence in Evaluation - Tessie Catsambas 
Thursday, Dec. 19, 2013

2-2:20 p.m. ET 

Recent attention on gender-based violence, a gender issue and a human rights violation, has shown it is so prevalent that, in some countries, an evaluator may come upon gender violence more than 25 percent of the time. Unprepared, the unsuspecting evaluator may misread the symptoms, draw irrelevant conclusions, and, in fact, contribute to more violence even when he or she is trying to help. This session will highlight methods and tools that an evaluator can use to identify gender violence if it is present, manage an evaluation touched by gender violence, and behave in a responsible way that protects victims. 

You can pre-register for these webinars by clicking the links above! 

____________________________________________________________________________________

Our eStudy program is made up of longer, more in-depth virtual professional development courses. Below are December's eStudy offerings:

eStudy 037: Essentials of Utilization-Focused Evaluation - Michael Quinn Patton

Dec. 9, Dec. 11, Dec. 13, and Dec. 16

2-3:30 p.m. ET

The evaluation standards call for evaluations to be useful, practical, accurate, ethical, and accountable. Utilization-Focused Evaluation (UFE) is a process that meets these criteria by promoting evaluation use from beginning to end with a focus on intended uses by intended users, and encouraging situational responsiveness and adaptability. In a series of four sessions, Michael Quinn Patton will present the basics, controversies, and cutting edge issues in conducting Utilization-Focused Evaluations. This eStudy workshop will be based on his latest book, "Essentials of Utilization-Focused Evaluation." 

Read more and register. 

• Research Associate at Harder+Company Community Research (Davis, Calif.)
• Research Associate / Evaluator at Office of Educational Innovation & Evaluation (OEIE) (Manhattan, Kan.)
• Programs & Evaluation Manager at United States Tennis Association (White Plains, N.Y.)
• Mat-Su Community Health Fellow at Mat-Su Health Foundation (Wasilla, Ark.)
• Research Associate II Social Science at The University of Memphis (Memphis)
• Evaluation Officer at Margaret A Cargill Philanthropies (Eden Prairie, Minn.)
• Executive Manager, District Assessment at Pinellas County Schools (Largo, Fla.)
• Senior Associate at Arroyo Research Services (Asheville, N.C.)
• Principal Investigator, MaxART Treatment as Prevention at Clinton Health Access Initiative (Mbabane, Swaziland)
• Impact Evaluation Advisor at DAI (Bethesda, Md.)

Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. Job hunting? The Career Center is an outstanding resource for posting your resume or position, or for finding your next employer, contractor, or employee. You can also sign up to receive notifications of new position postings via email or RSS feed.

• Register for an eStudy Web Workshop
• Sign up for one of the free (to members) Coffee Break Webinars

• Sign up for AEA's Electronic Discussion List EVALTALK
• Join the AEA LinkedIn Group 
• Download any of the 500+ Materials Available in AEA's eLibrary
• Follow AEA's Evaluation Headlines and Resources Twitter Feed 
• Visit AEA's Newsletter Archive for past articles of interest 

• Improve evaluation practices and methods.
• Increase evaluation use.
• Promote evaluation as a profession.
• Support the contribution of evaluation to the generation of theory and knowledge about effective human action.