Banner600
In this issue...
  • EPTF Update
  • Summer Institute
  • Robert Ingle
  • Graduate Interns
  • Call for Vendors
  • Program Eval
  • Research Logic
  • Find An Evaluator
  • EES Biennial Conference

  • AEA Newsletter
    May 2008

    AEA Colleagues,

    "Thank you." I don't have enough opportunities to say it face to face, but I am deeply thankful to those who are passionate about evaluation, about service, and about AEA. I have even more reasons than usual to say thank you to so many in the AEA community today.

    A record 1250+ conference proposals made for a busy winter as proposers submitted, staff cleaned, TIG chairs co-ordinated, and hundreds of members read and reviewed. I am in awe at the number of people who step forward each year to contribute their knowledge and expertise, their time and talents, so that the association may have an unforgettable program at the annual conference. You know who you are - those furiously writing to finalize and perfect a proposal, those spending nights reading, weekends on the phone, and breaks exchanging emails. Thank you for all that you contribute to the association's success.

    Those who weren't reviewing (and some who were) have been diving into the findings from the internal scan. Since March, every committee has looked at the data and preliminary reports, suggested further analyses, and considered how the story within may inform our work. Thank you to the over 2000 members who responded, providing insight into the membership and guidance for AEA's strategic thinking. Thank you to the volunteer leaders who moved us from analysis to interpretation and from interpretation to action.

    What is on the horizon? This month we will open up the housing bureau for the annual conference and focus on getting sessions scheduled for November. Conference proposal dispensation notices go out July 3, the same day as the opening of conference registration. Our contractor has compiled the first round of analyses and updated the reports from the internal scan and we will have them uploaded for member access this week - as soon as I figure out the best way to make 700 pages of appendices as accessible as possible! Finally, read below to learn about the first steps in our pilot program to draw upon learning circles as a professional development avenue for our mid-career members, and stay tuned for further updates about changes emerging out of the knowledge gained from the internal scan and the strategic planning processes.

    This is an exciting time to be working with AEA. Thank you for being part of our community of practice.

    Susan

    Susan Kistler
    AEA Executive Director



    EPTF Update
    AEA meets with federal work group to discuss evaluation practice

    An AEA task force continues to make progress in its efforts to encourage the federal government to establish sound policies for evaluating its programs. AEA President and Evaluation Policy Task Force (EPTF) Chair William Trochim, and AEA Executive Director Susan Kistler, met with a work group of the Office of Management and Budget's Performance Improvement Council (PIC). The PIC is helping to create an evaluation process for federal programs that is more ingrained within agencies. Trochim was the featured speaker at the meeting, which focused on integrating the evaluation process throughout a program's life cycle and on choosing evaluation methods that are appropriate to the stage of that cycle.

    "The work group was very receptive to hearing ideas from the evaluation community," Trochim said. "Crafting a government-wide evaluation process is an ambitious endeavor, and we want to encourage the group to base any such effort on sound policies and practices."

    "We went into this meeting discussing ways to improve impact evaluation throughout the federal government and many in the group found the discussion to be informative," said Katherine Dawes, who is a member of the EPTF as well as co-chair of the PIC work group. "After reflecting on our background material, including information shared at the meeting, the work group plans to address the development and identification of products that will help in the selection of appropriate evaluation designs and methods for a broad range of impact evaluation, as well as to develop proposals to revise current guiding documents."

    The PIC was established through an executive order issued in November by President Bush. A recent article in Government Executive magazine said Bush's order was prompted by a desire to leave behind a legacy of program performance improvement. "The desire of the current administration to leave a legacy that incorporates evaluation into all federal programs is admirable," noted Kistler. "We want to help contribute to that legacy being a positive one, that allows for nuance in the use of appropriate evaluation designs and methods matched to the program type, context, and development stage. Our hope is to develop an ongoing working relationship, spanning administrations, with those working on the ground in the federal arena."


    Summer Institute
    Annual event growing in numbers and scope

    This year marks the 8th Annual Centers for Disease Control and Prevention (CDC) Summer Evaluation Institute - and its third year co-sponsored by AEA. The event will draw more than 600 registrants this year, a significant increase in both the number and the diversity of participants. "In 2002, there were 127 participants but 75% were from CDC and almost none were from state, local, and community-based public health partners," notes Tom Chapel, who is both an AEA member and the CDC co-chair for the Institute. "By 2005, participation more than doubled. As importantly, more than half were from state, local, and community public health partners. In 2006, the first year of our AEA partnership, attendance doubled yet again to almost 600 people, and the crowd included many participants from areas other than public health. Participants that year told us in their evaluations that this diversity added value to their experience, allowing for cross-talk and opportunities to learn from each other."

    The AEA/CDC Summer Evaluation Institute is aimed at professionals who conduct or manage evaluations as well as those who use results for program improvement. Evaluators from any level of government, staff of nonprofit and community organizations, applied researchers, grant makers, foundation program officers, and social science students are encouraged to attend. While CDC is a sponsor, the focus is not exclusively on public health and evaluators working in any context will find relevant content. This year's event will include three keynote addresses, three rounds of three-hour training workshops, two rotations of 90-minute breakout sessions, plus two group lunches to allow for networking opportunities among attendees.

    If you are considering attending, please register at your earliest convenience. The 2008 AEA/CDC Summer Evaluation Institute has a limited capacity, registration closes once all slots are filled, and individual sessions are filled on a first-come basis. The event offers more than 50 evaluation-focused training sessions and will be held at the Sheraton Atlanta Hotel from June 23-25. Two workshops that provide an introduction to evaluation also are available to attendees on Sunday, June 22.

    Go to the Summer Evaluation Institute website

    Robert Ingle
    Distinguished Service Award

    Ingle. Guttentag. Myrdal. Lazarsfeld. Each name is tied to one of AEA's awards. Guided by the Awards Committee, we are researching the background and history behind each naming. Over the next few issues, we will share with you our discoveries. Today we begin with the Ingle Award, presented to a member who has been particularly instrumental in promoting AEA's interests and operations, and given for sustained and valuable service to AEA as an organization.

    AEA's Distinguished Service Award was named in honor of Robert B. Ingle. Lovingly called a curmudgeon by those who knew him best, Ingle received the award in 1985 after close to a decade of service first with the Evaluation Network and then with the Evaluation Research Society. The two groups merged that same year to become the American Evaluation Association and for ten more years Ingle continued his service by coordinating AEA's annual conference, laying the foundation for a legacy of success at the association's signature event.

    Viewed as a constant presence during a time of great change, "Bob was extremely helpful in the predecessor organizations and then to AEA itself," says AEA member John McLaughlin, a friend and colleague who helped found the Ingle Award. "He was the face of the national meetings and he was more than willing to go out of his way to ensure the success of the fledgling enterprise and to nurture its development. The current prominence of AEA is in part predicated on the work of Robert B. Ingle."

    When Ingle received the award himself, it was noted that even as the event grew from year to year, "the cast for organizing the conferences changes annually. In this way new ideas and improvements can be tried with the confidence that the administration of the conference and its program are in good and stable hands." Ingle died in 1998 at the age of 72.

    Credited with foundational contributions to the associations supporting the newly emerging discipline of program evaluation, and noted for his devotion and dedication to E-Net and AEA, Ingle personified the service ethic reflected in the award named in his honor.

    Go to the Call for Awards Nominations page

    Graduate Interns
    Partnership with Duquesne provides professional development opportunities

    Take a handful of graduate students with solid research backgrounds, add generous helpings of evaluation training and mentoring, and stir in seminars, workshops and real-world evaluation projects. That's the recipe used in the Graduate Education Diversity Internship Program, a partnership between AEA and Duquesne University. The program provides professional development opportunities for masters and doctoral students from populations that are traditionally under-represented within evaluation, including students of color, LGBT, and those with disabilities.

    Applications for the fifth cohort of interns are available on-line at http://www.education.duq.edu/mastersCertification/ diversity_inter_program.htm. To be considered for the program, completed application forms must be received by August 1, 2008.

    "This program promotes diversity within evaluation in several ways," says Dr. Rodney Hopson, Director of the program since it began in 2004 and a professor at Duquesne University. "It not only recruits graduate students from under-represented populations and extends their research capacities to evaluation, but it also stimulates evaluation thinking about these populations and deepens the field's capacity to work in racially, ethnically and culturally diverse settings."

    In the fall, interns will take part in an intensive three-day seminar featuring guest experts and senior evaluation colleagues who guide them in lectures, readings, discussions and assignments. They will also attend AEA's annual conference in Denver in November to participate in conference sessions as well as pre- and post-conference workshops. Two other workshops in the course of the internship will provide another 40 hours of exchange and learning.

    Meanwhile, interns have the opportunity to provide support to evaluation activities of an agency located near their graduate institutions. Throughout the 9-month internship, students, staff and mentors communicate mainly through the Google Groups website, where they post assignments, readings and questions and share their challenges and successes.

    AEA is seeking a director to guide the program through its next three-year period, which will run from academic year 2009-2010 until academic year 2011-2012. Persons interested in the post can get more information online at http://www.eval.org/aea08.internship.htm.


    Call for Vendors
    Expanding our use of "learning circles"

    A growing body of literature indicates that alternative models of professional development such as learning circles, communities of practice and/or learning communities are more effective than traditional training at building leadership and management related knowledge and skills as well as strengthening networks among seasoned practitioners (Dufour, Dufour, Eaker & Many, 2006; Collay, Dunlap, Enloe & Gagnon, 1998). Once beyond the basic technical knowledge and skill intensive induction phase, many mid-career learners benefit most from sustained participation in interactive, inquiry-based, multiple session opportunities that are focused on their specific issues and interests.

    AEA is undertaking a trial program that will employ learning circles as a professional development tool for mid-career evaluation professionals. A learning circle (LC) is a focused discussion group that harnesses the wisdom and experience of professional peers. Typically, a LC is issue focused and convenes around topics of interest to the circle participants. A facilitator hosts the group providing structure and a framework for high quality professional development that emphasizes peer learning and mentoring. LCs have the advantage over traditional professional development in that they provide consistent, deep focus that supports problem-solving, asset development, tool generation and/or exchange as well as spreading knowledge grounded in shared experience and practical application. We are at the beginning phase of this initiative, seeking a contractor experienced in providing training around LCs and this is where you come in.

    Would you help us to identify potential vendors who provide training in Learning Circles to add to our pool to receive the request for proposals?

    You can do this by sharing the below link to our Call for Proposals to Facilitate AEA's Learning Circle Experiential Training, which provides details on the program. Alternatively, you can send an e-mail directly to AEA Executive Director Susan Kistler at [email protected] with the name and contact information of any person or firm you believe meets the criteria set forth in the Call so that we may contact them directly.

    Stay tuned over the summer as we distribute information about how to apply to participate in the first round of LCs and how to become a circle facilitator. Stay tuned next winter to learn about the circles that will be offered throughout the association.

    Go to the Call for Proposals Page

    spauldingcover Program Eval
    New book examines real-life examples

    AEA member Dean T. Spaulding is author of Program Evaluation in Practice: Core Concepts and Examples for Discussion and Analysis. Released this winter from Jossey-Bass, the book features 11 evaluation projects and the real-life situations in which the evaluators find themselves.

    From the publisher:
    "The lack of teaching cases in program evaluation is often cited as a gap in the field. This ground-breaking book fills this gap, covering the essentials of program evaluation as it is used in education and with a wide variety of evaluation projects to be discussed, analyzed, and reflected upon. Individual cases cover classroom instruction, community-based program, teacher training, professional development, a secondary-school based program, after-school program, reading achievement, school-improvement grant, and confidentiality. Each case is structured to include learning objectives, program description, evaluation plan, summary of evaluation activities and findings, key concepts, discussion questions, class activities, and suggested reading.

    Designed as a practical classroom text, each of the book's case studies uses the following framework:

    • The Evaluator: Discusses the role of the evaluator in a particular project as well as the evaluator's background and education.
    • The Evaluation Plan: Maps out the evaluator's detailed plan and explains the objectives, methods, and tools that are used.
    • Summary of Evaluation Activities and Findings: Describes the evaluator's data collection process and offers an overview of the evaluation findings. At the end of each case, the evaluator is presented with a dilemma to resolve.
    • Final Thoughts: Concludes the chapter with an explanation of what really happened at the end of the evaluation, shows how the evaluator handled the dilemma, and provides the results of the project."

    "I think evaluators will be able to relate to these situations," says Spaulding. "I wrote Program Evaluation in Practice because I couldn't find a case study book out there in program evaluation. And if I did find examples they didn't seem to have some of the elementary components I was looking for."

    The author is an associate professor in the Department of Educational Psychology at the College of Saint Rose and is the former chair of AEA's Teaching of Evaluation TIG.

    Jossey-Bass Publishing offers AEA members a special savings on its publications when ordered directly from the publisher. To receive your 20% discount, please use the promotional code "AEAf8" online or by phone (1-800-225-5945).

    Go to the Publisher's Website

    Johnsoncover Research Logic
    Revised book provides step-by-step coverage of key elements

    AEA member R. Burke Johnson, a professor at the University of South Alabama, is co-author of a new book that outlines the multitude of research methods used in education today. Educational Research: Qualitative, Qualitative, and Mixed Approaches (3rd ed.) introduces readers to the fundamental logic of empirical research and the sources of research ideas. Detailed descriptions guide graduate and post- graduate students through the design and implementation of actual research studies with a balanced examination of quantitative, qualitative, and mixed research. The book is accompanied by a web- based study site and an instructor's resources CD.

    Produced by SAGE, the book provides step-by-step coverage of the key elements of research, including sampling techniques, ethical considerations, data collection methods, measurement, judging validity, experimental and non-experimental methods, descriptive and inferential statistics, qualitative data analysis, and report preparation. The book also describes the essential skills of conducting and evaluating research; using strong sampling designs; writing proposals and questionnaires; understanding quantitative, qualitative, and mixed methods research; and analyzing qualitative and quantitative data.

    Dr. Johnson has been an AEA member since 1992. At Evaluation 2007, he chaired a multi-paper session on issues related to randomized trials in educational evaluation. Specifics can be found at http://www.eval.or g/search07/session.asp? sessionid=8598&presenterid=957

    AEA members receive a 20 percent discount on books from SAGE publications when ordered directly from the publisher. The discount code for AEA members is S05CAES or members can call the Customer Care department at 1.800.818.7243

    Go to the Publisher's Website

    Find An Evaluator
    Get the most out of your membership experience

    Are you a full- or part-time evaluation consultant? Do you work in a firm or organization that provides evaluation services including capacity building, training, or conducting evaluations? AEA's "Find an Evaluator" service receives over 1000 unique pageviews each month and as a member you are entitled to a free listing on AEA's Find an Evaluator webpage. When the office receives inquiries from those searching for an evaluation practitioner, this is the site to which they are directed.

    Listings are short, focusing on basic services available from the evaluator or firm. In order to allow for more detailed exploration of a member's or firm's background, capacity, and services, each listing must link to a website. If you do not yet have a website, we have compiled a list of resources for developing a low-cost or no-cost site to promote your consulting work or firm. Listings are searchable by keyword, state, or country, or users may browse the listings and view them all at once.

    Over 500 members are included in our find an evaluator listings. Take a look at the listings to get an idea of examples and join them at your convenience. You will need to sign on to the AEA website using your username and password in order to post or update your listing, but the browsing and search functions are available publicly. Please consider supporting your AEA colleagues by encouraging use of this service when you are asked about finding an evaluator.

    Go to the AEA Find an Evaluator Webpage

    EES Biennial Conference
    Building for the Future in Lisbon

    The 2008 Biennial Conference of the European Evaluation Society (EES) will be held October 1 through 3 at the Lisboa Congress Center in Lisbon, Portugal. The theme of the conference is Building for the Future: Evaluation in Governance, Development and Progress. The conference, which is open to non-EES members as well as EES members and employees of EES Institutional members, will explore the relationship between evaluation use and the development of social capital. A series of issue-based strands will cover the range of considerations that evaluators, commissioners and practitioners have to make in order to embed evaluation in governance and development in ways that encourage and supports progress.

    "We see evaluation as one of the ways to help a society work more effectively and fairly," said the EES Board of Directors. "Evaluation can help build 'social capital' by increasing social cohesion and connectedness. In this understanding social capital is all about strengthening social networks, improving institutions and adapting procedures, - key issues for evaluators today." The conference will be preceded by professional development training sessions led by international experts on Tuesday, September 30 and Wednesday, October 1.

    Registration is now open!

    Go to the EES Conference Website
    AEA Site Links
  • AEA Home
  • TIGs
  • Affiliates
  • Board
  • About Us

    The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.

    The American Evaluation Association's mission is to:

    • Improve evaluation practices and methods
    • Increase evaluation use
    • Promote evaluation as a profession and
    • Support the contribution of evaluation to the generation of theory and knowledge about effective human action.

    phone: 1-508-748-3326 or 1-888-232-2275