|
|
| Newsletter: January 2010 |
Vol 10, Issue 1 |
|
|
|
|
|
| Welcome 2010 - A Year of Evaluation Quality |
 Dear AEA Colleagues,
Happy New Year! This is my first letter as President of AEA. It gives me quite a thrill to say that! I am looking forward to a year of talking to people about my presidential theme of Evaluation Quality, spreading the word about the value of AEA as a professional community and of evaluation as a practice, and supporting our efforts to provide high quality services that respond to member needs and interests. For example, one of our recent initiatives is the "coffee break" demonstration series - 20-minute webinars highlighting a specific tool or task. If you haven't submitted a proposal for AEA's annual conference before, watch for the February 4th demonstration on proposal submission! (For a complete listing of the demonstrations, go to http://www.eval.org/demos.asp.)
I am deeply honored to represent AEA as president, and I value the opportunity to lead a discussion about the quality of our work, but I also find the job of president somewhat daunting. One of the less visible roles of president is that of chair of the Board of Directors. Our 2009 President, Debra Rog, set a high standard for what the Board can accomplish in a year. Last year, with Debra's leadership, the Board:
- completed a full set of policies to guide the Association;
- organized Board committees into Priority Area Teams to assist in policy development in the areas of public engagement, values, leadership, knowledge and professional support, and finance; and
- identified potential revisions to the by-laws to make them "lean and mean" so that we have more flexibility to respond to changes in our environment.
(To review and comment on the current policies, please go to http://www.eval.org/policies.asp. More information about the proposed by-laws revisions will be coming out in the next couple of months with opportunity for your input.)
The emphasis on member input and engagement is one of the most exciting aspects of the policy-based governance model that the Board is using. I am committed to continuing and expanding the opportunities for member input that have recently been instituted and to ensuring that the Board uses that input in our decision-making. The other aspect of policy governance that I hope to champion is the systematic monitoring and evaluation of the Board's decisions and the association's activities. With the talent and leadership of my colleagues on the Board, I am confident that we will have another productive year.
I welcome your comments, ideas, concerns, and questions, and look forward to making many new connections this year. Thank you for your involvement in AEA!
Leslie
Leslie Cooksy
2010 AEA President |
|
 |
|
|
|
| Policy Watch - Pillars of Evaluation Policy |
|
From George Grob, Consultant to the Evaluation Policy Task Force
One of the difficult challenges any professional organization faces in trying to influence legislative, regulatory, and administrative policies is to articulate a clear and consistent set of cardinal goals or values that are of the utmost importance to the professional practice of its members and the field in which they work. On the surface, that might seem easy to do. In reality it is difficult to come up with a list of "policy pillars" which capture the essence of a profession, can be easily remembered, and can be used as points of reference from which to derive appropriate responses to myriads of elaborate proposals advanced by legislators and other government policy makers.
AEA's Board of Directors, with input from the Evaluation Policy Task Force (EPTF), took on this task at its November meeting during the AEA national conference. Following are the common pillars they adopted. The Board has authorized the EPTF and me to promote them in meetings and communications with outside groups involved in formulation of evaluation policies.
AEA Pillars of Evaluation Policy
Broad use of evaluation in public programs, especially those of the Federal Government
Using methods appropriate to the evaluation questions
Adapting the size and scope of evaluations to be appropriate to the program's context and needs
Adequate funding for evaluation
Use of qualified, experienced evaluators as appropriate
Evaluator independence
Transparency of results
|
| Mathematica Honored with AEA's 2009 Outstanding Evaluation Award |
|
 The American Evaluation Association honored Mathematica Policy Research as the recipient of its 2009 Outstanding Evaluation Award. Christopher Trenholm and Barbara Devaney were recognized at an awards ceremony held during AEA's Evaluation 2009 conference in Orlando, Fl. The New Jersey-based firm's work has led to major changes in federal policy, federal funding and has resulted in more than 100,000 downloads of the study itself.
"AEA's Outstanding Evaluation Award aims to improve evaluation practice and use by recognizing studies that can serve as models in the future," says Trenholm. "Its acknowledgment of Mathematica's abstinence education evaluation, which used the most rigorous, scientifically based approach to measure program impacts, is a high honor. I am pleased to accept this recognition of Mathematica's contribution to model research and to informing important public policies."
Mathematica was tapped in 1998 to spearhead a congressionally-mandated study of abstinence-only education programs intended to reduce teen pregnancies as well as sexually transmitted diseases. The comprehensive nine-year study - undertaken within a politically contentious arena - was noted for its rigor, balance, and impact. The study involved a multitude of stakeholders with varying views and interests, incorporated three rounds of follow up as much as six years after the program, had an 80 percent response rate over time, and ensured strict confidentiality of approximately 2000 teenage participants. The study has been widely cited in news articles and opinion pieces in the Washington Post, USA Today, NPR, BBC and ABC News. Newsweek's Sharon Begley in an article entitled Just Say No to Bad Science pointed to the evaluation as a model of good research in a field where it had been sorely lacking.
Mathematica's work and influence was noted for its depth of detail, the intricate involvement of diverse constituents and a long-term highly-confidential protocol that protected participants and heightened the study's success.
"I am proud to be the recipient of AEA's Outstanding Evaluation Award and would like to thank the organization for this honor," says Devaney. "The study illustrates how high quality research can dramatically redefine debate and decision-making on controversial, critically important policy issues. I would also like to acknowledge the teamwork of colleagues and partners that made this recognition possible."
Other recipients of AEA's 2009 awards will be profiled in future issues.
Go to AEA's Awards Page |
| AEA's 2009 Newsletter Click-throughs Indicate Resource Interests |
Readership statistics reveal AEA members utilize its monthly member newsletter at a rate almost double the industry standard - and the number of click-throughs for 2009 is 20% higher than in the year before. Readership trends also indicate that AEA newsletter readers take an avid interest in the newest reports, documents and tools and we consistently see a high interest in news regarding developments in evaluation policy, new job postings in AEA's Career Center and, of course, news regarding AEA's awards recipients. See below for a look at one indicator of what AEA members are reading:
Top AEA Newsletter Click-throughs for 2009
- What Constitutes Strong Evidence (referenced in 2008 Top 10 articles)
-
-
-
-
-
-
-
-
-
Please note that click-throughs are only one indicator of interest. Often an article stands on its own without clicking through to the content and often people are already familiar with the specific content and thus do not click through since they already are informed or can access it via an alternative avenue. However, in the interest of identifying indicators of success, these numbers provide one way of gauging member interest and use of AEA resources.
We would like to thank our contributors and readers for generating ideas, providing content and taking an avid interest in the life of the association. As always, we invite you to send suggestions to Gwen Newman, Newsletter Editor at gwen@eval.org. And, for those who would like to access past issues of the newsletter, you may always do so at the AEA newsletter archive.
|
| Ricardo Millett to Headline Thought Leaders Series in February |
 You've had the opportunity to connect with Michael Patton, Jody Fitzpatrick, and E. Jane Davdison via the AEA Thought Leaders Discussion Series. Next up? Ricardo Millett will be sharing his 40 years of lessons learned and insights the week of February 7-12, 2010.
Ricardo Millett is an independent evaluation consultant, a veteran philanthropist, Board member at the Center for Effective Philanthropy, former President of the Woods Fund of Chicago, and former Director of Evaluation at the WK Kellogg Foundation. Millett has been a consistent and prominent voice on issues of diversity and the intersection of evaluation, philanthropy, and social justice. As a senior consultant to the Diversity in Philanthropy Project (DPP), he was the primary author of a recent DPP Case Study on Evaluation With a Diversity Lens: Exploring Its Functions and Utility to Inform Philanthropic Effectiveness. Millett's commitment to making evaluation practical, useful, and integrated with programs has formed a cornerstone of his years of work in the field.
As a reminder, the Thought Leader Discussion Series brings together members for an informal week-long asynchronous discussion around issues raised by a guest thought leader. The exchange takes place online via the President's forum and postings may be checked at your convenience each day throughout the week.
How does one sign up for the AEA Discussion Series? This is a members-only opportunity and it is free to all members who wish to participate. You should be available to check the discussion list postings daily during the week of February 7-12. The full exchange takes place electronically via the AEA Presidents Forum. To join the forum, sign on to the AEA website using your username and password, navigate over to the forums, and click on "Add/Change Subscriptions" near the top right of the forum page.
Login at: http://www.eval.org
Your username:
Your password:
If you have questions about the forums, or run into technical difficulties, please check the Thought Leaders Frequently Asked Questions and then do not hesitate to contact us here in the AEA office at office@eval.org. |
| TechTalk - New Daily Blog, AEA365, Offers New Resources for Evaluators |
| From LaMarcus Bolton, AEA Technology Director
This past month, I've had the pleasure to chat with John LaVelle, the coordinator for the new AEA365 blog. In this Q&A, he'll explain a little more about AEA365, its benefits, and how to become involved.
What exactly is AEA365?
AEA365 is a daily blog that is dedicated to helping evaluators share the Hot Tips, Cool Tricks, and Rad Resources that they use in the course of their work. The blog was launched on January 1 of this year, and our goal is to feature the tips, tricks, and resources - plus the ensuing discussion via ongoing comments - from a different evaluator every day of 2010. There is so much knowledge and wisdom within our community, and this is another way of helping us get involved, connect, learn, and (hopefully) improve our practice!
What types of "hot tips, cool tricks, and rad resources" are featured?
Right now, we have a range of tips, tricks, and resources queued to be posted. For example, many evaluators use logic modeling as a way of generating understanding about the programs they're evaluating, and so we'll be featuring some high-tech and low-tech methods of developing logic models with your stakeholders. Other topics include technology for both PC and Apple computers, strategies for developing evaluation capacity building, guides & checklists, and a wide range of resources on displaying information. Of course, this is only scratching the surface of the topics we plan on covering. The content really is defined by what our members wish to contribute!
How do you find your content? We've been using multiple strategies for soliciting content for the blog. To help get the project started, we've been utilizing our personal networks of evaluators, and reaching out to individuals that presented on interesting topics at Evaluation 2009. In addition, we've reached out to the TIGs and the TIGs will be acting as guest curators and nominating some of their members to submit posts on a thematic topic. For example, we hope to see a series of posts that highlight tips, tricks, and resources related to qualitative methods, or resources related to evaluation in foundations. But, there is a lot of knowledge in the AEA community and a lot of subject matter experts, so if you or someone you know has a tip, trick, or resource that you think is interesting, send me an email: aea365@eval.org.
How does one subscribe? There are a number of ways to access the blog, you just have to find the one that's right for you! One way is by going to the webpage to see what's being shared, and if you're looking for specific information, you can search by topic area (e.g. logic models, GIS, checklists, etc). Alternatively, you can sign up to have the post of the day delivered to your email account, or via an RSS feed. All these options are available at the blog's main page.
If you have any questions about AEA365, please contact John at aea365@eval.org. Go to the aea365 homepage to read, subscribe, and learn more |
| AEA's Newest Topical Interest Group Focuses on Evaluation Policy |
AEA welcomes its newest topical interest group - the Evaluation Policy TIG. The purpose of this TIG is to invite inquiry into the nature, scope, structure, purpose, direction, consequences, and impact of evaluation policies. Emphasis will be on examining both formal and informal evaluation policies that influence and shape evaluation practices in public, private, and non-profit sectors. Evaluation policies affecting programs, organizations, professional disciplines, and fields of service delivery within each of these sectors will be explored through TIG discussions and activities. A particular focus of this TIG is to promote evaluation as a profession and deepen understanding of the field of evaluation through encouraging theoretical and empirical inquiry and investigation in the following areas:
-
Documenting and reporting upon existing evaluation policies and the implications these policies have on evaluation practice within the United States as well as internationally;
-
Collaborating to develop and test coherent evaluation policy frameworks;
-
Exploring the extent to which evaluation policies are adopted, implemented and differentially assessed in various contexts, and critically examining contextual and methodological factors that promote and/or undermine successful adoption of evaluation policies to strengthen evaluation practice;
-
Examining how evaluation policy may be developed or improved to support better decision making through integrated social problem analysis; trend, probable consequence, and alternative analyses; and strategic evaluation;
-
Examining how evaluation policy frameworks may be adapted and strategically utilized to facilitate evaluation mainstreaming, enhance existing evaluation infrastructure, and promote evaluation systems of inquiry to strengthen individual, organizational, service-sector, and field-based learning, development and "return on vision";
-
Considering how evaluation policies not only influence standards for judging the methodological rigor, utilization, and ethical implications of single evaluation studies, but also discussing appropriate standards for judging how an entire policy research/evaluation system contributes to improved policy making for social betterment.
The TIG's ultimate aim is to invite multidisciplinary inquiry that expands and deepens understanding of evaluation policy and its influence and impact in advancing the quality, utilization, integrity, and professionalism of evaluation practice; and conversely, of evaluation practice and how varying evaluation practices are used to inform and develop evaluation policy. The Evaluation Policy TIG plans to i) engage AEA members in identifying and clarifying evaluation policy issues; ii) facilitate constructive dialogue around diverse interests, values, and objectives relevant to evaluation policy; iii) promote better understanding of political, technical, and social processes and contextual issues bearing upon the interconnectedness of evaluation policy and evaluation policy; iv) create an investigative and reflective forum that encourages and supports individual and collective responsibility for the development and consequential impact of professional evaluation as manifest through evaluation policies and practices.
If you would like more information about the Evaluation Policy TIG, contact TIG Chair Lisa Rajigah at lrajigah@3ieimpact.org. To join the TIG online, sign in using your AEA username and password. You can then update your member profile to include the TIG as one of your five TIG memberships.
Your username is:
Your password is: |
| American Journal of Evaluation and the Arbor Day Foundation |
AEA will be partnering with SAGE, publisher of the American Journal of Evaluation, and the Arbor Day Foundation, in 2010 to donate a tree for every association member who opts not to receive a print copy, and instead receive online-only delivery of AJE.
AEA and SAGE are working to decrease the environmental impact of association publishing, while ensuring that you still have access to essential journal content. Through the members-only section of the AEA website, all members have access to AJE online, as well as online access to New Directions for Evaluation, Evaluation and the Health Professions, and Evaluation Review. The online content from each journal goes back over 20 years and is highly searchable to help you find just the article that will move your practice and thinking forward.
The Arbor Day Foundation is a nonprofit conservation and education organization. Founded in 1972, The Arbor Day Foundation has grown to become the largest nonprofit membership organization dedicated to planting trees, with nearly one million members, supporters, and partners. The Arbor Day Foundation plants trees that are appropriate to the region in which they are working and engages individuals and communities to plant and learn about our trees and forests. SAGE and AEA are proud to be partnering with the Arbor Day Foundation starting in 2010 to plant trees and help reduce our collective carbon footprint.
If you have not already done so, please consider joining over 500 of your AEA colleagues in receiving AJE in online-only format. To do so, update your membership profile by signing in to the AEA website at http://www.eval.org or send Heidi in the AEA office an email, at heidi@eval.org, noting "please sign me up for AJE online-only." Together, we can plant a forest!
Go to the Arbor Day Foundation Website to Learn More About Their Work |
| Report to AEA on IOCE's 2009 Annual General Meeting |
By Jim Rugh, AEA Representative to IOCE
 In December, the International Organization for Cooperation in Evaluation (IOCE) held its Annual General Meeting (AGM) using two virtual media: a live webinar, followed by active email discussions during the following week.
The Board acknowledged the need to do a better job of keeping IOCE members and others informed of what IOCE and other international evaluation organizations are doing. With help from volunteers, a periodic e-newsletter will be distributed to representatives of all 97 evaluation associations on the IOCE's database. Also, thanks to an initiative by Marco Segone of UNICEF, an information hub using web 2.0 technologies will facilitate shared communications as well as searching for evaluation-related resources from many sources.
One of the main purposes of the IOCE is to share information about and among evaluation organizations globally, and to promote the formation and strengthening of evaluation associations. Towards that end, the existing 14 case studies documenting the development and operation of evaluation associations will be updated and expanded.
IOCE will be coordinating a survey of standards being used by evaluation associations around the world, as well as initiatives by some groups to develop credentials for our profession. It would be useful to know what some organizations have developed or are working on.
Though IOCE represents the '4th network' (organizations of evaluation professionals, especially those in developing countries) as part of the Network of Networks on Impact Evaluation (NONIE), there is concern that Nonie's Guidelines for Impact Evaluation do not adequately represent the diversity of perspectives and methodologies. IOCE intends to address this through proposing revisions or additions to the NONIE Guidelines.
Ross Conner (previous AEA Representative to the IOCE and past IOCE President) reminded us of the critical role of IOCE in making wise choices of priority initiatives, given its limited resources. He proposed an innovative approach, promoting partnerships among various associations to work together on joint projects.
Reinforcing Ross's point, we want to extend an invitation to any AEA members who are interested in participating in any of these initiatives to let us know by sending me an email at JimRugh@mindspring.com. Certainly AEA as an organization, or any AEA affiliate, in addition to individual AEA members, have much to share with - and learn from - our partner organizations and professional colleagues around the world.
Finally, it was agreed that the terms of Oumoul Ba Tall as President and Burt Perrin as Vice President should be extended until the end of 2010, in order to allow more time for transition to new leadership. In concluding the AGM, Oumoul Ba Tall thanked the participants, observing that this had been the most active AGM IOCE has held. This shows the possibilities of more collaboration using various virtual media. She also pledged that IOCE will do all it can to promote the interests of the global evaluation profession.
Go to the IOCE homepage to learn more |
| Many Ways to Contribute to AEA - Webinars, Bloggers, etc. |
|
Looking for ways to get involved in the life of the association? AEA's new Member Involvement Initiative (MII) has the following updates related to new volunteer opportunities within AEA:
Planning Team for Webinar Series: AEA began a series of short, free, webinars as of January 1. By this time next year, we hope to build upon these initial offerings and expand our webinar-based content to include longer fee-based training. Towards that end, we hope to learn more from attendees, other webinar providers, TIG leaders, and the membership at large to inform planning related to appropriate content, delivery, and structures. Would you like to be considered to serve as a member of the planning team? We're seeking members willing to make a one-year commitment of no more than 1 hour per week and who meet one or more of the following criteria: (a) active in TIG program planning; (b) ongoing participants in the current webinar series; (c) background in distance-based learning; and/or (e) experience with other groups offering webinar-based content. If you are interested, please send an email by February 15 to Susan Kistler in the AEA office at susan@eval.org noting your background and interest.
AEA365 Blog Contributors: The aea365 blog is dedicated to highlighting Hot Tips, Cool Tricks, and Rad Resources for evaluators. Begun on January 1, 2010, our goal is to feature a post a day from and for evaluators around the globe. We're seeking colleagues to write one-time contributions of 250-400 words sharing tips, tricks and/or resources from their own experience. No blog writing experience is necessary - you simply review examples on the aea365 blog site, craft your entry according to the contributions guidelines, and send it to John LaVelle our blog coordinator. He'll do a final edit and upload. If you have questions, or want to learn more, please review the site and then contact John at aea365@eval.org. |
|
New Jobs & RFPs from the AEA Career Center |
|
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:
- Associate Director at The Evaluators' Institute, George Washington University (Washington, DC, USA)
- Senior Research Analyst at Utah Valley University (Orem, UT, USA)
- Visiting Project Coordinator at University of Illinois at Urbana-Champaign (Champaign, IL, USA)
- Evaluation Consultant at Michigan Public Health Institute (Lansing, MI, USA)
- Director of Programs Anne Ray Charitable Trust at Margaret A. Cargill Philanthropies (Wayzata, MN, USA)
- Client & Business Development Specialist at Edvantia Inc. (Nashville, TN, USA; Charleston, WV, USA)
- Science & Technology Analyst at Science and Technology Policy Institute (Washington, DC, USA)
- Research Associate at Randi Korn & Associates (San Francisco, CA, USA)
- Research Assistant Level 2 at Ciurczak & Company Inc. (Buffalo, NY, USA)
- Lead Evaluator at National Democratic Institute (Bosnia; Washington, DC, USA; Telecommute)
Descriptions for each of these positions, and many others, are available in the AEA Online Career Center. According to Google analytics, the Career Center received over to 4500 unique visits in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee. Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed. Go to the AEA Online Career Center |
| Get Involved |
| Get the most from your membership by taking advantage of the many things that you can do right now to participate in the life of the association, share your input, and promote your business.
|
| About Us |
| The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275
| |
|
|
|
|
|
|
|
|
