|Newsletter: July 2011||Vol 11, Issue 7|
| Expanding the Scope & Reach of Evaluation News|
Wow! So much is happening this summer, so much news. From the usual scandals involving high-profile people (Dominique Strauss-Kahn), to the last shuttle flight on Atlantis, to the intractable budget "debate" in the U.S. government, to the Japanese victory in the thrilling final of the Women's World Cup Soccer tournament, to news about the integrity of news itself as media mogul Rupert Murdoch's empire crashes on allegations of illegal phone hacking and bribing of police.
Here are some news items from within AEA. First, as briefly reported in the last newsletter, AEA's board continued its transition to a policy-based form of governance by convening in mid-June a one-day retreat of Priority Area Team (PAT) members and other member leaders of the association. The primary purpose of the retreat was to develop principles and structures for meaningful member engagement in AEA policy activities. The group was congenial and lively, we had frank conversations and worked hard, and the PATs are now taking the next step of recommending future structures and processes for member engagement in AEA's policy work. We look forward to hearing their good thinking. Second, two AEA empirical projects are currently underway. You recently received an invitation to provide input to the International Listening Project from Jim Rugh, our contractor for this project. JVA Consulting in Denver, Colorado has initiated an evaluation of AEA's transition to policy governance and is likely to be soliciting input from members in the near future. And third, our November conference program is taking exciting shape as notices for over 1300 proposal acceptances were sent out earlier in July.
These are forward-looking times for our association. And I am hoping that we can begin to be part of the news out in the world, congruent with our Goals Policies for evaluation users and the general public. We have a modest history of newsworthy evaluation work. I believe it is time to build on that history, educate our clients and the general policy-shaping community about the central importance of evaluation to wise decision making, and begin to claim some of the headlines. Initial efforts in these directions include a draft statement for the general public on 'what is evaluation' (stay tuned) and the ongoing work of the Evaluation Policy Task Force. Forward evaluation!
AEA President, 2011
|Policy Watch - The Roadmap Gets Legs|
|From George Grob, Consultant to the Evaluation Policy Task Force
AEA published its most recent version of An Evaluation Roadmap for a More Effective Government in September 2010. AEA and the Evaluation Policy Task Force have used it in commenting on emerging national policies such as maternal, infant, and children's home-visiting programs, health care reform, and foreign assistance. A broader goal was to use it as AEA's "place at the table" when policies were being considered, even if an AEA representative couldn't actually be there in person.
The latter goal is now being realized. The Roadmap has been cited in congressional testimony, Government Accountability Office (GAO) reports, and other settings. AEA's Executive Director Susan Kistler recently posted on the AEA website the results of her search for citations to the Roadmap in public documents. Here is a brief summary.
Criteria for Effective Research and Evaluation Programs
Four GAO reports cited the Roadmap as the criteria against which the adequacy of Federal agencies' research and evaluation programs could be judged: the McGovern-Dole Food for Education program, USAID's international feeding program, Afghanistan drug control programs, and the Department of Labor's Employment and Training program.
Establishing and Prioritizing Research and Evaluation Plans
GAO cited the Roadmap in a report evaluating the methods used by Federal departments to plan their research programs. An IBM Center for the Business of Government blog, commenting on this GAO, further emphasized the value of the Roadmap as a useful reference on this topic.
The Need for Federal Program Evaluation and A Framework for Institutionalizing It
NASA, in its report on informal education, cited the Roadmap as establishing the need for evaluation and providing a framework for developing an effective evaluation program. Similarly, the Welcome Trust Project quoted the Roadmap at length regarding systemic and systematic evaluation of Federal programs. The Economist's Intelligence Unit cited the Roadmap in making its point about the need to fully embrace and fund evaluation of Federal programs.
Avoiding the Over-Emphasis on a Single Method for Evaluation
The Brookings Institution Center on Children and the National Institute for Early Education Research, in a report on early childhood policy, used language in the Roadmap to emphasize the need for more than randomized clinical trials to evaluate program interventions. Grantmakers for Effective Organizations, in its briefing paper, Scaling What Works, cited the Roadmap as a basis for emphasizing the context of program decision making in choosing the most appropriate evaluation approaches and methods. Similarly, Deborah Daro, in her testimony before the House Ways and Means Subcommittee on Income Security and Family Support, quoted the Roadmap in emphasizing the need to integrate monitoring and evaluation with program management and also to avoid reliance on a single evaluation approach.
Now that the Roadmap is catching on, those of you with connections to Federal Government agencies might want to share it with your contacts. Maybe it will go viral. If you are aware of any citations, please send them to [email protected].
Go to AEA's Evaluation Policy Task Force website page
|AEA's International Listening Project Launched|
|From Jim Rugh, International Listening Project Coordinator|
On July 7, an invitational letter from AEA President Jennifer Greene and myself was sent to the AEA membership, as well as to colleagues in other evaluation associations around the world. This launched the first phase of the International Listening Project (ILP) called for by the AEA Board as it seeks input to strategies for enhancing our engagement with evaluators, evaluation associations, and evaluation clients around the globe. As Jennifer pointed out in her introduction, the Board sees this initiative to be aligned with AEA's mission and values, specifically reflecting AEA's commitment to
valuing a global and international evaluation community and understanding of evaluation practices.
As coordinator of the ILP, I want to remind you that there are several modalities for submitting your suggestions:
- The primary instrument for soliciting individual input is an online questionnaire. We invite you to go there now and submit your suggestions, if you haven't done so already. It only takes 10-15 minutes, depending on how creatively and extensively you respond to the open-ended questions.
- Another modality for sharing is an interactive blog where multiple persons can offer their suggestions and comment on those of others. After you respond to the SurveyGizmo questionnaire, we invite you to go to the Wikispaces blog where you can join the discussions. Check out the various pages and comments already posted and feel free to add your own.
- Another mode would be to send your personal comments directly to me at [email protected].
We solicit your contributions in whatever form you choose. In order to accommodate those of you who may be motivated by this e-newsletter to respond to the survey, the deadline for the first round of input has been extended to August 5.
An initial synthesis of some of the input received has already been posted to the Wikispaces blog, where participants have opportunities to provide feedback and contribute to the process of prioritizing the list of policy recommendations and potential activities. A draft summary report will be presented to the AEA Board by the end of August and it is anticipated that a final Board-approved report of the International Listening Project will be completed and disseminated by mid-October. There will then be opportunity for further direct involvement by members during a Think Tank discussion to be held Wednesday, November 2, from 4:30-6:30 p.m., at Evaluation 2011 in Anaheim, California.
Jim will be AEA's Thought Leader for the week of August 7-13, focusing on issues related to AEA's place and presence in the international arena. During the discussion forum, Jim will share preliminary survey results and welcomes member input.
Go to AEA's Thought Leaders Discussion Series Web Page
|Graduate Education Diversity Internship Program Graduates Eight|
AEA's Graduate Education Diversity Internship (GEDI) Program graduated its seventh cohort of interns during June's AEA/CDC Summer Institute in Atlanta. Attendance at the Institute capped off a nine-month internship experience which included placement at an evaluation agency, the production of an evaluation report, attendance at two 4-day seminars as well as the AEA annual conference, and numerous assignments throughout the year designed to maximize their learning experience.
"Chelle and I are truly grateful to have had the opportunity to work with such an incredible group of students, and are quite pleased with the personal and professional growth they experienced over the course of the year," says Rita O'Sullivan, GEDI program co-chair (with Michelle Jay). Jay added, "The program's structure - laid by the program's founder, Dr. Rodney Hopson - continues to provide them with powerful learning opportunities designed to deepen their understanding of the field, of the vital importance of culturally responsive evaluation practice, and, most importantly, of how much the field and the organization has to gain from the knowledge, skills, and experiences the interns bring to the table."
Please join us in congratulating this year's graduating class! The 2010-2011 GEDI graduates are:
LeKisha M. Harris, University of Georgia
LeKisha interned with ICF Macro (Atlanta, Georgia), where she and her colleagues conducted evaluability assessments of initiatives considered to be "promising practices" in the areas of nutrition, physical activity, and obesity prevention.
Dawn Henderson, North Carolina State University
Dawn interned at Evaluation, Assessment and Policy Connection (Chapel Hill, NC), where she participated in an evaluation, sponsored by the National Y-USA, that sought to measure the program effectiveness of 40 YMCA programs.
Alison Mendoza, The University of North Carolina at Chapel Hill
Alison also interned at Evaluation, Assessment and Policy Connection, where she participated in process evaluation efforts around the Y-USA Educational Achievement Initiative.
Ebun Odeneye, University of Texas - Houston
Ebun interned at The University of Texas Prevention Research Center, where she participated in the formative evaluation of a collaborative effort between several tribal organizations to adapt and evaluate an evidence-based teen pregnancy/STI prevention program for middle-school students.
Yusuf Ransome, Columbia University
Yusuf interned with The Conversation Company (TCC) Group (New York), where he worked on multiple evaluation projects, but the bulk of his work was associated with the evaluation of Moishe House.
Christopher St. Vil, Howard University
Chris interned at Westat (Rockville, MD), where he participated in the evaluation of WFF's High Need Family (HNF) program, a permanent supportive housing program for homeless families.
Tamara Williams, University of Colorado-Boulder
Tamara interned with Community Health (Boulder, CO) where she conducted a needs assessment of the organization's existing sexual education content in terms of its relevance to female students of color.
Ciara Zachary, Johns Hopkins Bloomberg School of Public Health
Ciara interned with the Annie E. Casey Foundation (Baltimore, MD), where she participated in an evaluation of Elev8, a national initiative dedicated to ensuring that middle school students succeed in life and continue on to high school.
Go to AEA's GEDI Program page to learn more
|TechTalk - View Missed Webinars in AEA's Coffee Break Archive |
From LaMarcus Bolton, AEA Technology Director
Did you know that as an AEA member you have free access to over 50 short recorded webinars from our Coffee Break Webinar Series? Each Coffee Break Webinar is first offered live, usually on Thursdays at 2 PM Eastern. The live session is recorded and the recordings are then available for streaming from AEA's members-only Coffee Break Webinar archive.
First, let me encourage you to attend the free real-time offerings whenever you can. Attending live means that you can ask questions important to your understanding. Coffee Break Webinars are each approximately 20 minutes in length - 10 minute presentations, followed by 2 minutes of housekeeping (introductions, reminder to complete evaluation form), and 8 minutes of questions. Attending live also means you'll receive the follow-up email from the moderator with links to any resources or other items mentioned during the webinar itself.
You can view the upcoming webinars online. We're finalizing the schedule now for late summer through fall with webinars on topics such as data management, using video, and analysis strategies. So keep your eyes open! New webinars are announced in the emailed calendar that comes your way or you can subscribe to AEA's Headlines and Resources weekly compilation where Stephanie Evergreen, our eLearning Director, posts registration notices.
I know sometimes you just can't attend a live session - even when it's a favorite topic. Maybe the timing is bad, other obligations keep you away, you have newly joined the association, or you recently acquired an interest in something new. The breadth of what's been offered is extensive and you can review the recordings right in your web browser.
Want to learn about Collaborative, Participatory, and Empowerment (CPE) Evaluation? The CPE TIG completed a series of three webinars in March to cover the basics.
Hoping to improve your reporting and presenting? Data visualization and reporting are among the most well-covered topics and John Nash's April 2010 presentation on Moving Beyond Bullets was among the very best reviewed.
You can also find Michael Patton speaking about Utilization-focused evaluation, SaraJoy Pond introducing you to Wordle for making word clouds, and Jennifer Camacho explaining the basics of nonparametric statistics - plus over 40 other topics and speakers.
So, dive into the webinar archive and check out all we have to offer. Explore the options and feel free to make suggestions. You can contact me directly at [email protected].
Go to the Archived Webinar List
|New Community Psychology TIG|
Please welcome AEA's newest TIG. This TIG is intended to provide a community of support and collaboration for members with a mutual interest in community psychology and/or community-based research, action and evaluation. The Community Psychology Topical Interest Group is designed for community psychologists and allied community-based researchers and practitioners who share a common set of principles and values that influences both the content of their work and the methodologies employed.
Though a number of AEA TIGs are relevant to community psychologists and AEA members with an interest in community research and action and share common perspectives and methodologies, none are inclusive of the many principles, goals, guiding concepts, and competencies that are inherent in community psychology/community based practice.
"There are a number of community psychology practitioners who work as evaluators. This TIG will provide a regular, formalized opportunity for those of us with similar disciplinary roots and training to gather and share our work," says TIG organizer and co-chair Susan Wolfe. "It will also provide a venue for us to share our discipline within the larger field."
Many community psychologists and community-based researchers and practitioners work in evaluation, but often evaluation is done within the context of a larger project, such as prevention program development, program implementation and/or organizational capacity building. Program evaluation is one of several competencies that community psychologists develop, including needs assessment, community organizing, collaboration building, capacity building, program development, logic modeling, translating research to practice, advocacy, and program dissemination.
The primary goals of the new Community Psychology TIG are to:
- Provide a forum for discussion and collaboration for evaluators with community psychology training and/or a similar disciplinary orientation
- Support professional development
- Promote evaluations guided by Community Psychology principles, goals, and guiding concepts.
- Provide a linkage mechanism between the Society for Community Research and Action (SCRA) and the American Evaluation Association (AEA) that would promote membership for both organizations.
For those interested in the Community Psychology TIG, you may email Susan directly at [email protected].
|AEA Welcomes Newest Local Affiliate|
Join us in welcoming the Texas Evaluation Network (TEN) as AEA's newest local affiliate.
The Texas Evaluation Network is established as a not-for-profit interdisciplinary organization for the purposes of promoting and advocating evaluation and building evaluation capacity; providing professional development opportunities for Texas evaluators; and establishing a venue for networking and the exchange of theoretical, methodological, and practical knowledge related to the field of evaluation. During Evaluation 2010, held last November in San Antonio, Texas, an exploratory meeting was convened to discuss the establishment of a statewide professional organization for evaluators. More than 30 individuals attended, representing a geographic cross-section of Texas evaluators as well as a variety of evaluation specialty areas such as education, health care, business, social services, international evaluation and independent consulting. It was agreed during the meeting that there was sufficient interest to create a Steering Committee to develop a framework for a new organization.
"The growing demand for evaluation expertise within our State has led us to pursue the establishment of the Texas Evaluation Network," says Steering Committee co-chairs Debra Stark and Ray Howard. "We believe that aligning this new organization with AEA not only strengthens its credibility but also provides the opportunity to collaborate with the AEA to promote and advocate for the evaluation profession and encourage opportunities for resource sharing, e.g. membership and event planning."
In the near future, TEN will be sending to all Texas AEA members specific information regarding the organization, its structure, and how to become a member. In the interim, you may visit their website or contact the co-chairs directly at [email protected] or [email protected].
Go to the TEN website
|In the News|
AEA member Paul R. Brandon of the University of Hawaii's College of Education Curriculum Research & Development Group (CRDG) was named the 2011 Research on Evaluation Distinguished Scholar by the American Educational Research Association (AERA). He is the first recipient of this newly created sanctioned award, which will be given annually by the AERA Research on Evaluation Special Interest Group. The formal presentation of the award occurred in New Orleans at AERA's annual meeting.
If you have a news item you'd like to submit for consideration, feel free to forward to AEA's Communications Director at [email protected].
|Southeast Evaluation Association Event Oct. 27-28|
The Southeast Evaluation Association will host its 23rd Annual Workshop, Biggest Bang for your Buck: Using Evaluation to Demonstrate Impact During Lean Times on Oct. 27-28 at Tallahassee Community College in Tallahassee, Florida.
The event includes a pre-workshop seminar by Liliana Rodriguez-Campos. Collaborative Evaluations Step-by-Step is offered as an interactive session that will share factors that facilitate and inhibit stakeholders' participation and address fundamental factors that must be mastered in order to succeed in collaborations. Liliana serves as Director of the Center for Research, Evaluation, Assessment, and Measurement at the University of South Florida and is the Program Chair of AEA's Collaborative & Empowerment Topical Interest Group as well as a recipient of AEA's Marcia Guttentag Award.
A Call for Proposals is open through August 31. SEA is seeking speakers on a broad array of program and policy areas at the national, state and local levels including public administration and management, health and human services, criminal justice/juvenile justice, economic and workforce development, education, and not-for-profit issues.
Go to the SEA Website
New Jobs & RFPs from AEA's Career Center
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:
- Field Coordinator/Analyst at Northern Illinois University (DeKalb, IL, USA)
- Sampling Consultant at World Vision International (USA and Any)
- Senior CQI Analyst at Community Behavioral Health (Philadelphia, PA, USA)
- Post-Graduate Fellow at Institute for Health Metrics and Evaluation (Seattle, WA, USA)
- Principal Management Auditor and Management Auditor at King County Auditor's Office (Seattle, WA, US)
- Director, Brand Evaluation Portfolio at LTG Associates, Inc. (Takoma Park, MD, USA)
- Waste Reduction Education Evaluation at Metro (Portland, OR, USA)
- Evaluation Research Assistant at SmartStart Educational Consulting Services (Irvine, CA, USA)
- Part-time Research Analyst at University of Connecticut (Storrs, CT, USA)
- Research Analyst-Evaluation at Westat (Rockville, MD, USA)
Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. According to Google analytics, the Career Center received approximately 6,300 unique page views in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee. Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed.
|The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275