|
|
| Newsletter: November 2010 | Vol 10, Issue 11
|
|
|
|
|
|
| Evaluation 2010 and Moving Forward |
 Dear Colleagues,
I hope everyone had a wonderful time in San Antonio at Evaluation 2010! I enjoyed seeing old friends and making new ones. Over 2500 people attended and the feedback I received was quite positive. (Remember to complete the conference evaluation if you haven't already!)
Prior to the conference, the AEA Board met and, in addition to our usual business, did some important planning for next year. Our usual business consisted of reviewing policies and planning evaluations. Specifically, we discussed policy recommendations from the Finance Priority Area Team regarding financial practices and the role of the Treasurer and from the Values Priority Area Team on the review of Guiding Principles. You can see the revised policies here - the additions are noted in highlights. We also received a report and policy recommendations from the Knowledge and Professional Support Priority Area Team for Topical Interest Groups. Those recommendations will be considered fully at the Board's January meeting. The evaluation planning focused on evaluations of the transition to policy-based governance and the Evaluation Policy Task Force (EPTF). The RFP for the evaluation of the transition is being finalized now, and we expect to make progress on the plan for the evaluation of the EPTF in January.
Looking to next year, the Board discussed how to move forward with a review of the Priority Area Team (PAT) structure and how to begin to develop policies on AEA's international presence and role. For the former, the Board is planning a one-day meeting with PAT members in mid-June to discuss appropriate structures for substantive AEA member involvement in policy setting and review. For the latter, a Board Task Force is developing a plan for a "listening campaign" to ensure that any policies considered are well-grounded in an understanding of what will be mutually beneficial and appropriate. As always, I hope you will feel free to share your thoughts on either of these issues - or any others - with me or any Board member.
It was great to see so many of you in San Antonio!
Best wishes,
Leslie
Leslie Cooksy
|
|
 |
|
|
|
| Policy Watch - AEA Comments on GAO Auditing Standards | |
From George Grob, Consultant to the Evaluation Policy Task Force
As many of you may know, the Government Accountability Office (GAO) is seeking public input on its sixth revision of the Generally Accepted Government Auditing Standards, commonly known as the Yellow Book. Because of the close connection of the Yellow Book to evaluation, AEA decided to respond. After consultation with AEA members and the AEA Board of Directors, AEA's Evaluation Policy Task Force prepared comments which were sent to GAO on November 22 under the joint signature of AEA's President, President-Elect, and Past President.
The Yellow Book has evolved in recent years, gradually encompassing studies called performance audits that overlap with evaluations. For example, the Yellow Book explicitly refers to AEA's Guiding Principles for Evaluators, and The Program Evaluation Standards put forward by the Joint Committee on Standards for Educational Evaluation.
AEA's comments noted the importance of the Yellow Book, especially to evaluators who conduct performance audits. However, AEA also noted "overly rigid requirements . . . that inappropriately constrain all performance audits within the mold of a normative methodology." The comments also call for greater emphasis of a standard of completeness, namely that "Being complete . . . also means the report states evidence and findings without omission of significant relevant information related to audit objectives."
Based on the limited response from AEA members to the invitation to comment (only one member submitted input) and from a panel discussion of the Yellow Book at the recent AEA conference, it appears that very few AEA members are familiar with the Yellow Book. Perhaps their being called "Auditing Standards" accounts for this. Some of us (including myself) have been more intimately involved with them, and as noted in the request for comments, two of our outgoing AEA Board members - Mike Hendricks and Rakesh Mohan - serve on GAO's Advisory Board for the Yellow Book (although not as representatives of AEA). Even though the Auditing Standards are not comprehensive in their treatment of evaluation related topics, they do contain useful advice. Particularly noteworthy are their treatments of various types of performance audits, evidence, conflict of interest, and threats to independence.
Hopefully GAO will react favorably to AEA's comments, but whether they do so or not, evaluators can learn some useful practices from auditors. The parts of the draft Yellow Book that may be of special interest to evaluators are:
Chapter 1 - Ethical Principles
Chapter 2 - Section on Performance Audits (pages 12-14)
Chapter 3 - General Standards
Chapter 6 - Field Work Standards for Performance Audits (paragraphs 6.15, 6.37-6.75)
Chapter 7 - Reporting Standards for Performance Audits
Appendix I - Additional Information (pages 166-167 and 180-187)
For those interested, here is wishing you all happy reading.
Go to the AEA Comments on the GAO Auditing Standards
|
| TechTalk - A Round-Up of AEA's Internet-Based Resources | |
From LaMarcus Bolton, AEA Technology Director
 At our most recent Evaluation 2010 conference, you may have noticed business cards floating around with tidbits of information. These weren't traditional business cards, but served to highlight some of our most popular Internet-based initiatives. I wanted to take a short break from our "Technology in Evaluation" series to briefly discuss how several of these mediums could be used to continue the flow of knowledge post-conference.
Webinar Demonstrations: Did you know that AEA offers both short "coffee break" and longer professional development webinars? Our coffee break webinar series offers members the opportunity to learn the basics of particular tools of use to evaluators, while the professional development webinars extend this concept to also include opportunities to engage with experts and ask questions in a highly interactive environment. Occasionally, conference and workshop presenters will give demonstrations to complement their respective sessions. If you would consider presenting, please email webinars@eval.org. But, for more information on attending and learning useful skills, please visit our webinars page. Aea365 Alerts: Our aea365 tip-a-day alerts are aimed at sharing resources, tips, and lessons learned from and for evaluators. The aea365 medium just so happens to be a great place to introduce a topic that could lead to a potential workshop or conference presentation, or to re-introduce a topic previously presented for more detail and/or additional clarification. If you would like to contribute, please email aea365@eval.org. Or, to stay abreast of these invaluable daily tips, please consider updates via email or RSS. Forums: AEA also offers several forums, including but not limited to: the Technology forum, New Member forum, TIG forums, and our very popular Thought Leaders forum. These forums allow members to contribute new and insightful posts, or simply 'continue the discussion' - which is of great relevance to our recent conference. For example, if you were intrigued by a session sponsored by the Quantitative Methods TIG, you could easily post follow-up thoughts and questions within their respective forums. Both members and presenters alike are able to login and start offering contributions at will! eLibrary: Lastly, I wanted to briefly discuss our eLibrary. The AEA eLibrary is a member-created knowledge management system for all evaluation-relevant information. The library holds all major file types, including Microsoft Office, PDF, audio, and document sets. The eLibrary is especially relevant to the recent conference, as it provides a medium for storing any supplemental information from presenters. This is especially helpful if one was unable to attend a session of interest, or if one just wanted an electronic copy of presentation materials. Presenters, I encourage you to upload your materials, if possible. And if you did not present at the conference, we still encourage you to browse and upload any materials of relevance. I truly hope everyone is able to take advantage of the above resources - they are truly phenomenal in their own unique ways. If you run into any issues or have any questions or suggestions, please do not hesitate to contact me at marcus@eval.org. Let's keep the flow of knowledge going! |
|
AEA Honors International Evaluator Jim Rugh with Evaluation Practice Award | |
Join us as we congratulate the recipients of this year's AEA Awards. Recognized on Friday, Nov. 12 in conjunction with AEA's Evaluation 2010 conference in San Antonio were four awardees who've helped heighten international evaluation efforts, spearhead a groundbreaking new journal, influence a health initiative that impacted the lives of children and families in five urban communities, and influence a new generation of evaluators. We will spotlight one recipient below and will highlight others in upcoming issues of AEA's monthly newsletter. Thank you all for your generous contributions to AEA, to the field, and to our greater communities.
 Jim Rugh, a veteran evaluator who helped found AEA's International and Cross-Cultural Evaluation Topical Interest Group and who serves as AEA's Representative to the International Organization for Cooperation in Evaluation (IOCE), was honored as the 2010 recipient of AEA's Alva and Gunnar Myrdal Evaluation Practice Award.
"Among the many factors recognized were his two decades of sustained service to the American Evaluation Association and to our field; his exceptional leadership in developing guidelines for helping communities to become more involved in evaluation; for developing evaluation capacity through his practice and for encouraging transparency and stakeholder involvement in his work," says Tarek Azzam, chair of AEA's 2010 Awards Committee.
"In important ways, Jim has been a global ambassador for evaluation for much of his career," says Jennifer Greene, AEA's President-Elect. "Building on his early work in international development, Jim has been instrumental in establishing and promoting evaluation in this sector for the last two decades. His work at CARE is especially notable, as it involved the development and dissemination of systems of monitoring and evaluation, to include staff training, meta-evaluation, and ongoing promotion of evaluation itself. Jim must have developed and nurtured strong, respectful relationships as well as credibility with his international partners during this time period. For, since then, Jim has been in high demand as an international evaluation practitioner and also a trainer. And since then, Jim has garnered respect and renown as an advocate for strong evaluation."
Rugh has been professionally involved for more than 45 years in rural community development in Africa, Asia, Appalachia and other parts of the world. He has specialized in international program evaluation for 30 years, including 12 years as head of Design, Monitoring and Evaluation for Accountability and Learning for CARE International, where he was responsible for promoting strategies for enhanced evaluation capacity throughout that worldwide non-governmental organization (NGO). He has also evaluated and provided advice for strengthening the monitoring and evaluation systems of a number of other international NGOs and is recognized as a leader in the international evaluation profession. Along with Michael Bamberger and Linda Mabry, Rugh was a coauthor of the popular book RealWorld Evaluation: Working under Budget, Time, Data and Political Constraints, published by Sage in 2006.
Though he retired from full-time work in 2007, Rugh continues his active involvement in the evaluation profession. This includes frequent international travel to lead training workshops and consultancies with a variety of international development agencies. Rugh is entering his fourth year as AEA's Representative to the IOCE (the collaborative network that Rugh refers to as "the UN of professional evaluation organizations"), where he also serves as Treasurer and an active member of the IOCE Executive Committee.
"It is truly an honor to be chosen to receive the 2010 Alva and Gunnar Myrdal Practice Award," says Rugh, an author, trainer and longtime evaluation practitioner who currently resides in Sevierville, Tennessee. "It is especially gratifying to receive this recognition for my career-long involvement in helping to promote evaluation capacity and evaluation organizations around the world - a cause that has been, and remains, not only my profession but also my passion." Rugh is a graduate of the University of Tennessee as well as Cornell University and is fluent in English, French and Hindi.
We invite you to think about AEA's 2011 Awards and to consider nominating colleagues for this distinction. You'll find an overview at the link below, as well as specific nominating instructions.
Go to AEA's Awards Page |
|
Meet AEA's Newest Members of the Board | |
In a previous issue, we introduced the results of our 2010 election and the names of our new incoming board members, AEA Treasurer and 2012 President. We promised you then that we would follow up with more specifics on each of them. In this issue, we spotlight our three incoming board members who will serve 2011-2013. Congrats again to Christina Christie, Jenny Jones and Victor Kuo. Read on to learn more about their unique backgrounds, insights and outlooks.
 Christina Christie is an Associate Professor in the Social Research Methods Division of the Graduate School of Education and Information Studies at the University of California, Los Angeles, where she co-directs its evaluation program. Tina served in the same capacity in her previous position on the faculty of Claremont Graduate University. In addition to her teaching, Tina conducts evaluation studies, research on evaluation practice, and theoretical analysis. She served two terms as chair of the Theories in Evaluation TIG and helped found the Research on Evaluation TIG. She also co-founded the Southern California Evaluation Association (SCEA), a local affiliate of AEA that offers local evaluators an opportunity to network and participate in an evaluation community while attending lectures, debates, and discussions on critical and timely evaluation issues. In 2004, she received AEA's Marcia Guttentag Early Career Achievement Award, given to a promising evaluation scholar during the first five years of his/her career after the completion of the dissertation.
 Jenny Jones is an Associate Professor in the School of Social Work at Virginia Commonwealth University and has been a member of AEA since 2001. She has served in numerous capacities within the organization to include: founding member of Social Work TTIG, Program Chair of the Social Work TIG, Ambassador, and Chair of the Diversity Committee, Task Force member for the Cultural Competency statement, Facilitator of the Asa Hillard Think Tank, and abstract and journal reviewer. Jones shares that her love for evaluation practice is rooted in the principles, practices, and her personal belief in social justice as a means for conducting research. She explains, "as a researcher, it is important that my perspectives regarding the population under study (families, communities and organizations) reflect fairness and justice for all; thus, evaluations that I conduct are guided by a culturally responsive framework specific to the population under study and performed in an efficient manner, without compromising the research." Evaluation practice allows for this to happen by assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to answer the ultimate question, "Does this work and how can it be improved"? Jones summarizes, "How exciting is that!"
 Victor Kuo is a Senior Research Associate in the Evaluation Research Program at WestEd. Most recently, he served as an evaluation officer for the Bill & Melinda Gates Foundation, and previously for the David & Lucile Packard Foundation and the James Irvine Foundation. "The positive impacts of, as well as challenges to, evaluation in philanthropy continue to attract attention and are now ubiquitous as topics of conversation and publication," says Kuo. "Throughout my career and still now, AEA is the source for evaluation professionals to learn about evaluation in foundations as well as in government, business, and industry sectors." AEA leadership roles include co-chair of the Nonprofit and Foundation TIG (2004-2006) and chair of the Professional Development Committee (2006).
|
| Face of AEA - Meet Leah Goldstein Moses | |
AEA's 6,300 members worldwide represent a range of backgrounds, specialties and interest areas. Join us as we profile a different member each month via our Questions and Answers column. This month's profile spotlights Leah Goldstein Moses, who interfaces with AEA most specifically through her association with a local affiliate.
 Name, Affiliation: Leah Goldstein Moses, President/CEO, The Improve Group. Member of Minnesota Evaluation Association and AEA
Degrees: Masters of Urban and Regional Planning with a concentration in evaluation, research and analysis methods (Humphrey Institute of Public Affairs, University of Minnesota)
Years in the Evaluation Field: 11
Joined AEA: 2005
AEA Leadership Includes: Past Communications Chair, MNEA
Why do you belong to AEA?
"AEA gives me the opportunity to learn from my colleagues. As an evaluation generalist, I am constantly "borrowing" methods used in one field (education, health, corrections, arts, technology) for use in another field. I love the chance to learn from specialists as well as problem-solve with other generalists. The Evaluation Consulting TIG has been invaluable, too, for everything from finding a great IRB to which recorders work best in focus groups."
Why do you choose to work in the field of evaluation?
"I love learning. Evaluation gives me the opportunity to learn something new every day, and to help others learn, too. When working with my clients (most of whom are not evaluators) I call myself an "evaluation cheerleader" - really, everyone has evaluative skills, and if their confidence and skills grow, they can benefit their organization."
What's the most memorable or meaningful evaluation that you have been a part of - and why?
"My favorite evaluation was with an organization called New Foundations. We used primarily qualitative methods - interviews and guided discussions. We got some great ideas and insights, and the organization put some of the insights to use immediately. Plus, the people they serve had great, touching stories to tell about the hard lives they lived and how the organization was helping them plan for a brighter future."
What advice would you give to those new to the field?
"Be excited! Experiment! Reach out for advice when you need it - but don't be afraid to innovate or try something new. We'd all like to learn from what you've tried."
If you know of someone who represents The Face of AEA, send recommendations to AEA's Communications Director, Gwen Newman, at gwen@eval.org. |
| The Handbook of Practical Program Evaluation | |
 AEA members Joseph S. Wholey, Harry P. Hatry and Kathryn E. Newcomer are co-authors of a new updated book released by Jossey-Bass. The Handbook of Practical Program Evaluation builds upon the success of the previous two versions with numerous contributors from the field.
From the Publisher's Website:
"More than ever before, the media, elected officials, and the public are demanding accountability from public and nonprofit organizations. But assessing the effectiveness of programs and services can be a time-consuming and costly process. In this comprehensive handbook, experts in the field of program evaluation outline efficient and economical methods of assessing program results and identifying ways to improve program performance. Written for managers, administrators, and educators in government, nonprofit, and private institutions, the authors of the Handbook of Practical Program Evaluation demystify and simplify the process of assessing program results. The authors describe the nuts and bolts of how to create an evaluation design and how to collect and analyze data in a way that will result in low cost and successful evaluations. For programs that require more complex evaluations, they offer instructions on advanced assessment techniques. The book explores the many approaches to program evaluation, including process evaluations, continuous monitoring of program performance, and ad hoc studies of past and present programs. This vital information can be used to aid in the decision-making process on budget alternatives, developing new legislative proposals, improving the performance of public and nonprofit organizations, and enhancing the overall management of public funds."
From the Authors:
"The publisher approached Harry Hatry, who recruited Joe Wholey, who recruited Kathy Newcomer for the first edition of the Handbook of Practical Program Evaluation. The Handbook of Practical Program Evaluation sold many copies, and the publisher then asked us to do a second and now a third edition of the handbook. We've been fortunate enough to recruit knowledgeable people as authors and co-authors, who produced excellent chapters on Evaluation Planning and Design, Data Collection, Data Analysis, and Use of Evaluation Findings. Handbook authors were willing to go through a number of drafts to provide clear, understandable, and practical advice within strict page limits. In the third edition of the handbook, we were able to include a number of chapters on new subjects, improving the handbook's usefulness. These new subjects include:
- Engaging and working with stakeholders
- Conducting exploratory evaluation
- Contracting for evaluations
- Using more kinds of qualitative methods, such as semi-structured interviewing and "collecting stories"
- Recruiting participants for, and retaining participants in, evaluation studies
- Using the Internet for evaluation studies
- Designing, managing, and analyzing multi-site evaluations
- Formulating options and recommendations for program improvement
- Understanding the politics of evaluation."
About the Authors:
Joseph S. Wholey is professor emeritus at the School of Policy, Planning, and Development at the University of Southern California. Harry P. Hatry is director of the Public Management Program at the Urban Institute, where he also previously served as director of the State-Local Government Research Program. Kathryn E. Newcomer is director of the Trachtenberg School of Public Policy and Public Administration at The George Washington University.
AEA members receive 20% off on all books purchased directly from Jossey Bass. Use the code "AEA10" when checking out from their website.
Go to the Publisher's Website
|
| Volunteer Opportunities - Outreach Working Group | Looking for ways to get involved in the life of the association? AEA's Member Involvement Initiative (MII) has the following updates related to new volunteer opportunities within AEA.
Outreach Working Group: The AEA Board has set as a priority for 2011 "outreach to evaluation users and the public to increase the value of evaluation to society." How should we go about defining the many publics and engaging them to improve the utility of evaluation and create a groundswell of demand? What do we want? Quality evaluation! When do we want it? Now!... And throughout a program's lifecycle. The OWG will identify opportunities for targeted public outreach, assess their likely impact, and recommend priorities for engagement in the near- and longer-term. If you have great ideas related to public outreach, a knowledge of the breadth of the field and its public stakeholders, and a passion for making a difference, send an email to Susan in the AEA office at susan@eval.org by December 16 with a short note indicating your interest and capacity in this area to be considered for the OWG. |
|
New Jobs & RFPs from AEA's Career Center | |
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:
- Associate Director of Research Services at Community Research Partners (Columbus, OH, USA)
- RFP - Public Policy Evaluator at Donors Forum (Chicago, IL, USA)
- Healthcare Program Evaluator at Free & Clear (Seattle, WA, USA)
- Senior Program Data Analyst at First Things First (Phoenix, AZ, USA)
- Research Assistant at National Center for Women & Information Technology (Boulder, CO, USA)
- Senior Program Officer - Global Development at Bill & Melinda Gates Foundation (Seattle, WA, USA)
- Research Analyst at The Policy & Research Group (New Orleans, LA, USA)
- RFP - External Evaluator for NSF Grant at Hagerstown Community College (Hagerstown, MD, USA)
- Evaluation Research Associate at Shore Research Inc. (Austin, TX, USA)
- Baseline Assessment Lead Evaluator, Nepal Democracy/Governance Program at National Democratic Institute for International Affairs (Washington, DC/Nepal)
Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. According to Google analytics, the Career Center received more than 12,660 unique page views in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee.
|
| Get Involved | | Get the most from your membership by taking advantage of the many things that you can do right now to participate in the life of the association, share your input, and promote your business.
|
| About Us |
| The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275
| |
|
|
|
|
|
|
|
|
