Banner600
In this issue...
  • Internal Scan
  • Notable Figures
  • WREN Meeting
  • Conference Hotels
  • Leadership Eval
  • New Affiliate
  • Milestones

  • AEA Newsletter
    June 2008

    AEA Colleagues,

    What a wonderfully busy time of year in the AEA office! The Internal Scan results are ready for distribution and I encourage you to dive in and see what we've learned about the membership - and to stay tuned as we share more about how we're acting on those findings. Earlier this month, we had the opportunity to build connections with colleagues in Washington at the meeting of the Washington Research Evaluation Network. And next week, AEA co-hosts the AEA/CDC Summer Evaluation Institute in Atlanta. In the meantime, the Board meets this Friday and Saturday in Denver. I am excited to share with them the beauty of the conference city and look forward to sharing with you more about the Board's work over the summer.

    We're also looking forward to November. The conference hotels have opened their room blocks and we're putting the final touches on the conference schedule before proposal status notices go out the first week in July. The program promises to be exciting. Plan ahead to explore the conference program when it goes online July 3. And, please, book your hotel early if you wish to get into any of the three currently contracted blocks. With record attendance expected in Denver, we are likely to fill the primary hotels well in advance of the conference.

    I wish you the very best of summer.

    Susan

    Susan Kistler
    AEA Executive Director



    Internal Scan
    View findings online

    The results are in and ready for viewing! The American Evaluation Association undertook an Internal Scan to learn more about its membership during the period September 2007 to January 2008. AEA contracted with Goodman Research Group, Inc. (GRG) to complete the scan and data collection. GRG surveyed the full membership, conducted follow-up interviews and online Question and Answer (Q&A) groups, and compiled and analyzed the results. The results from this scan are being used to guide strategic and operational decision making throughout the association.

    We are now working to disseminate the results from that scan in a number of ways. On the scan webpage, you can find a comprehensive report, a PowerPoint presentation, qualitative and quantitative indexes of analyses, and the instruments themselves.

    Our analysis and dissemination work is not complete. The first open-ended question of the survey generated hundreds of pages of qualitative responses. We are diving into that rich content further, as well as conducting additional analyses based on requests from the ten AEA standing committees currently reviewing the scan.

    At the upcoming conference, we will offer three separate sessions focusing on the scan.

    1. 1. The first will focus on the basic findings and the ways in which AEA is acting on them.
    2. 2. The second will focus on the process and methodology used in conducting the scan.
    3. 3. The third will focus on the extensive data from the remaining open-ended question, elaborating on what it means to be an evaluator.

    Over the coming months, we will use the newsletter to share findings of particular interest, as well as to keep you informed about the related sessions at the conference and next steps based on the scan's findings.

    While the documents shared online include an extensive set of analyses, suggestions for other analyses of the Internal Scan data may be sent to Susan Kistler in the AEA office at [email protected]. If you have any questions, please do not hesitate to forward to the same address.

    Go to the Internal Scan webpage to download reports and instruments

    daviesphoto Notable Figures
    Meet Rick Davies from M&E News

    As part of AEA's commitment to help keep members aware of global developments in evaluation, we are launching a series of interviews with notable figures within the international evaluation community. Our first interview is with Rick Davies, whose "Monitoring and Evaluation News" website ( http://www.mande.co.uk) and its associated email list connect evaluators in far-flung countries and offer links to a wealth of evaluation resources. An independent consultant based in the United Kingdom (UK), Rick also developed the now widely-used tool known as "Most Significant Change" (MSC). Below are some excerpts from the interview:

    "The email list now has more than 1,700 members worldwide, which makes it one of the largest in the field. It also has a noticeably high proportion of members from Africa and Asia, which is great to see...

    "Overall, I continue to be surprised how helpful people on the email list can be to others, especially when some don't seem to have done much prior 'homework!' The more interesting and longer term challenge is to identify where the email list and website best fit within an increasingly large and complex ecology of other websites and email lists dealing with monitoring and evaluation issues..."

    "[One development that concerns me] has been the increasing proportion of bilateral aid that is being spent via Direct Budget Support (DBS) to governments that are believed to have some minimal level of competence... My experience with the Ghana Poverty Reduction Strategy has highlighted how limited government capacities can be, how hard it can be to develop that capacity, and how limited donors' interests are in a critical review of that capacity. For many international donors, the use of DBS is administratively convenient and ideologically comfortable. This limits the type of evaluation findings that are likely to be listened to..."

    "Lack of transparency can be an indicator of ineffectiveness, in that those responsible for corrupt and / or ineffective practices are not usually keen for these practices to be made publicly visible. On the other hand, the presence of transparency can be an instrument for increasing effectiveness, because, as they say in the world of software design, 'with enough eyeballs, all bugs [problems] are shallow...'"

    "...(A)id projects must not only have an impact and be able to identify that impact. They must also be able to communicate to others how to replicate that impact."

    For more information about Rick and his perspectives on evaluation, the complete interview is available on the AEA website. Thanks to members of AEA's International Committee for their assistance with this interview.

    Go to full online interview

    WREN Meeting
    Connecting with research evaluators

    Evaluation is always a topic of great interest in the nation's capital, and AEA is increasingly an important part of the discussion. The association worked with the Washington Research Evaluation Network (WREN) to support and learn from a June 6 conference at George Washington University that brought evaluators together with policymakers, program managers, and evaluation managers from science and technology organizations.

    AEA President Bill Trochim delivered the keynote address, in which he described the work of the association's Evaluation Policy Task Force that is encouraging federal agencies to incorporate sound evaluation policies in their efforts to increase accountability. He also took part in a discussion panel at the day-long conference.

    "This conference was a bit of an experiment around AEA's engagement in the Washington area," Trochim said. "Working with AEA members associated with the federal government, we hope that we might be able to develop a variety of mechanisms that will enhance our ability to influence federal evaluation policies."

    Gretchen Jordan, chairman of AEA's Research, Technology, and Development Evaluation TIG, said her group and WREN had been working jointly in this area for the last several years - with the TIG holding academic/practitioner-oriented meetings and WREN hosting international workshops for government policy makers and program managers.

    Research in this area is now being funded by the National Science Foundation's "Science of Science and Innovation Policy" (SciSIP) initiative, and their announcements describe the sorts of methodology development for which AEA members can apply for funding. New methodologies developed by SciSIP will be presented for discussion at the next WREN meeting in December this year.

    Evaluation is also receiving substantial attention from a White House Office of Science and Technology Policy (OSTP) initiative on the "Science of Science Policy. An OSTP Science of Science Policy task force comprising evaluation representatives from federal R&D agencies is developing a "road map" of research, data, and data tools needed to develop this science. A preliminary draft of the road map is expected sometime this summer and will be a major topic at the next WREN meeting.

    "We're still in the beginning phases of coming up with the best tools for evaluating R&D," said Cheryl Oros, coordinator of the conference and a member of WREN's Steering Committee as well as the OSTP Science of Science Policy workgroup. "It's important that we bring everybody together to see if anybody in federal agencies can allocate resources for this and sponsor pilot evaluations using new tools, perhaps teaming on efforts. How do we talk the policymakers into investing in these tools? We see AEA as an important part of this process."

    Go to the WREN Wiki for the meeting proceedings

    Conference Hotels
    Room blocks now available

    The conference room blocks are now open for Evaluation 2008! Although the conference seems to be a long way off, the hotel room blocks will fill well in advance of the event. If you will be attending the conference, and wish to stay at your first choice hotel, please make reservations at your earliest convenience. Rooms will be filled on a first-come, first-served basis and rooms outside of the blocks are at considerably higher prices.

    We hold blocks at three hotels, each with its own charm:

    • The Hyatt Regency is the headquarters hotel for the event. New and modern, this four diamond hotel offers outstanding service right in the heart of the event. Approximately 2/3 of the conference sessions will be held here with the other 1/3 across the street at the convention center. Rates at the Hyatt start at $159.
    • The Hilton Garden Inn is next door to the Hyatt and is a great affordable option with a 24 hour complimentary business center. The rooms are freshly renovated, and complimentary internet rounds out the package. Rates start at $129.
    • The Magnolia Hotel is a modern boutique property two blocks from the Hyatt with complimentary internet, cookies, and breakfast! The updated rooms and soaking tubs, and comfy public spaces make it a fun alternative. Rates start at $140.

    The pre-conference workshops run from Monday, November 3, through 3:00 pm on Wednesday, November 5. The conference itself begins on Wednesday at 3:15 and runs through Saturday, November 8, at 5:30 pm. Post-conference workshops round out the event on Sunday, November 9, from 9:00 am to 12:00 pm. We'll be telling you more about the wonderful things to see in and around Denver in upcoming newsletter articles. Make a week of it and see all that Colorado has to offer! The hotels have a limited number of rooms available on the weekend before the conference begins and on the Sunday evening after it ends.

    Take a moment to review the hotel descriptions and insider tips online at http://www.eval.org/eval2008/08hotels2.htm. There you will also find details on how to book into the discounted conference blocks.


    hannummartineaubookresized Leadership Eval
    Impact evaluation the CCL way

    AEA members Kelly Hannum and Jennifer Martineau have co-authored their third book together. Evaluating the Impact of Leadership Development is published by Pfeiffer Publishing.

    From the publisher's website:
    Evaluating the Impact of Leadership Development is a step-by-step guidebook for creating and implementing evaluation of leadership development systems. Approaching issues from an evaluative perspective enables leadership development professionals to consider multiple perspectives and draw lessons as a natural part of the way work is done.

    Hannum and Martineau have worked together for the past 10 years. Hannum is an enterprise associate at the Center for Creative Leadership (CCL). Her work focuses on evaluation and leadership development across different cultures. Martineau is director of CCL's Design and Evaluation Center. Her work focuses on the evaluation of leadership development programs and initiatives. CCL is a top-ranked, global provider of executive education that develops better leaders through its exclusive focus on leadership education and research. Hannum and Martineau joined CCL in 1993.

    "Evaluation is often still thought of as an external process, something that can be dealt with AFTER a program has taken place," says Hannum. "We wanted to provide a perspective on evaluation that highlighted it as a positive and ongoing process -- for clarifying purposes, articulating different perspectives, and learning how to effectively and efficiently reach shared, or at least agreed upon, goals."

    Hannum served as chair of AEA's Business and Industry Topical Interest Group from 2005-2007 and Martineau is a member and former chair of AEA's Awards Committee.

    "Leadership development evaluation is a complex undertaking and there is still much to learn," says Martineau, "but it was nice to have a project that prompts us to reflect and to document what we currently know in a way that is hopefully of value to others."

    Go to CCL's Online Bookstore

    New Affiliate
    Arkansas group marks AEA's 25th

    The Arkansas Group of Evaluators (AGES), AEA's newest local affiliate, has been meeting and providing training opportunities since 2005. Designed to build evaluation capacity throughout the state by sharing evaluation information and tools, AGES emerged from the tobacco prevention work sponsored by the Minority Initiative Sub-Recipient Grant at the University of Arkansas at Pine Bluff. In the process of collecting tobacco prevention data, it became apparent there was a need to build evaluation capacity throughout the state. AGES is now formally linked to the new Arkansas Evaluation Center and the evaluation certificate program at the University of Arkansas at Pine Bluff.

    AGES provides an educational complement to these programs, offering specific training opportunities and is a vehicle to help with socialization of new evaluators to the field. AGES has hosted evaluation training workshops in empowerment evaluation, experimental design, quantitative methods, qualitative methods, focus groups, and interviews. In addition, educational technology workshops have been provided with a focus on online surveys, blogs, web pages, videoconferencing on the net, and telephony. AGES and the Arkansas Evaluation Center are sponsoring a free summer workshop on July 10-11.

    The group is AEA's 25th local affiliate and was organized by its chair, Linda Delaney and David Fetterman, board member and past president of AEA. The group operates on a calendar year with elections held in January. Officers are in place for a three-year term and terms are renewable. For more information, contact the chair at [email protected].

    Go to the Arkansas Group of Evaluators Blog Spot

    Milestones
    Changes in the evaluation community

    Congratulations!

    Ray C. Rist, Advisor to the World Bank, has been elected to a three year term (2008-2011) as President of the International Development Evaluation Association (IDEAS.) IDEAS is the international professional association for those working in and contributing to development evaluation. IDEAS has approximately 700 members in 95 countries. IDEAS has a ten member board with six members from developing countries and four from developed countries. Functioning largely as a virtual community, IDEAS emphasizes learning and sharing via technology. IDEAS was organized in Beijing, China in 2002 and held its first global assembly in New Delhi in 2004. Subsequently, it has worked with a number of other evaluation associations to co-host meetings and conferences. Its two most recent events were to co- host the Malaysian Evaluation Society meeting in March 2008 and the Global Environment Fund meeting in Alexandria, Egypt in May 2008.

    2007 AEA Award Winners
    The names of the 2007 AEA award winners are now online and permanently archived via an article recently published in the American Journal of Evaluation, Volume 29 Issue 2. Congrats again to Arlen Gullickson, Karen Kirkhart and Liliana Rodriguez-Campos. As an AEA member, you have free access to all current and back content from AJE. Just sign on using your username and password to the members only section of the website at
    http://www.eval.org/.

    AEA Site Links
  • AEA Home
  • TIGs
  • Affiliates
  • Board
  • About Us

    The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.

    The American Evaluation Association's mission is to:

    • Improve evaluation practices and methods
    • Increase evaluation use
    • Promote evaluation as a profession and
    • Support the contribution of evaluation to the generation of theory and knowledge about effective human action.

    phone: 1-508-748-3326 or 1-888-232-2275