The results are in and ready for viewing! The American Evaluation Association undertook an Internal Scan to learn more about its membership during the period September 2007 to January 2008. AEA contracted with Goodman Research Group, Inc. (GRG) to complete the scan and data collection. GRG surveyed the full membership, conducted follow-up interviews and online Question and Answer (Q&A) groups, and compiled and analyzed the results. The results from this scan are being used to guide strategic and operational decision making throughout the association.

We are now working to disseminate the results from that scan in a number of ways. On the scan webpage, you can find a comprehensive report, a PowerPoint presentation, qualitative and quantitative indexes of analyses, and the instruments themselves.

Our analysis and dissemination work is not complete. The first open-ended question of the survey generated hundreds of pages of qualitative responses. We are diving into that rich content further, as well as conducting additional analyses based on requests from the ten AEA standing committees currently reviewing the scan.

At the upcoming conference, we will offer three separate sessions focusing on the scan.

1. 1. The first will focus on the basic findings and the ways in which AEA is acting on them.
2. 2. The second will focus on the process and methodology used in conducting the scan.
3. 3. The third will focus on the extensive data from the remaining open-ended question, elaborating on what it means to be an evaluator.

Over the coming months, we will use the newsletter to share findings of particular interest, as well as to keep you informed about the related sessions at the conference and next steps based on the scan's findings.

Evaluation is always a topic of great interest in the nation's capital, and AEA is increasingly an important part of the discussion. The association worked with the Washington Research Evaluation Network (WREN) to support and learn from a June 6 conference at George Washington University that brought evaluators together with policymakers, program managers, and evaluation managers from science and technology organizations.

AEA President Bill Trochim delivered the keynote address, in which he described the work of the association's Evaluation Policy Task Force that is encouraging federal agencies to incorporate sound evaluation policies in their efforts to increase accountability. He also took part in a discussion panel at the day-long conference.

"This conference was a bit of an experiment around AEA's engagement in the Washington area," Trochim said. "Working with AEA members associated with the federal government, we hope that we might be able to develop a variety of mechanisms that will enhance our ability to influence federal evaluation policies."

Gretchen Jordan, chairman of AEA's Research, Technology, and Development Evaluation TIG, said her group and WREN had been working jointly in this area for the last several years - with the TIG holding academic/practitioner-oriented meetings and WREN hosting international workshops for government policy makers and program managers.

Research in this area is now being funded by the National Science Foundation's "Science of Science and Innovation Policy" (SciSIP) initiative, and their announcements describe the sorts of methodology development for which AEA members can apply for funding. New methodologies developed by SciSIP will be presented for discussion at the next WREN meeting in December this year.

Evaluation is also receiving substantial attention from a White House Office of Science and Technology Policy (OSTP) initiative on the "Science of Science Policy. An OSTP Science of Science Policy task force comprising evaluation representatives from federal R&D agencies is developing a "road map" of research, data, and data tools needed to develop this science. A preliminary draft of the road map is expected sometime this summer and will be a major topic at the next WREN meeting.

"We're still in the beginning phases of coming up with the best tools for evaluating R&D," said Cheryl Oros, coordinator of the conference and a member of WREN's Steering Committee as well as the OSTP Science of Science Policy workgroup. "It's important that we bring everybody together to see if anybody in federal agencies can allocate resources for this and sponsor pilot evaluations using new tools, perhaps teaming on efforts. How do we talk the policymakers into investing in these tools? We see AEA as an important part of this process."

The conference room blocks are now open for Evaluation 2008! Although the conference seems to be a long way off, the hotel room blocks will fill well in advance of the event. If you will be attending the conference, and wish to stay at your first choice hotel, please make reservations at your earliest convenience. Rooms will be filled on a first-come, first-served basis and rooms outside of the blocks are at considerably higher prices.

We hold blocks at three hotels, each with its own charm:

• The Hyatt Regency is the headquarters hotel for the event. New and modern, this four diamond hotel offers outstanding service right in the heart of the event. Approximately 2/3 of the conference sessions will be held here with the other 1/3 across the street at the convention center. Rates at the Hyatt start at $159.
• The Hilton Garden Inn is next door to the Hyatt and is a great affordable option with a 24 hour complimentary business center. The rooms are freshly renovated, and complimentary internet rounds out the package. Rates start at $129.
• The Magnolia Hotel is a modern boutique property two blocks from the Hyatt with complimentary internet, cookies, and breakfast! The updated rooms and soaking tubs, and comfy public spaces make it a fun alternative. Rates start at $140.

The pre-conference workshops run from Monday, November 3, through 3:00 pm on Wednesday, November 5. The conference itself begins on Wednesday at 3:15 and runs through Saturday, November 8, at 5:30 pm. Post-conference workshops round out the event on Sunday, November 9, from 9:00 am to 12:00 pm. We'll be telling you more about the wonderful things to see in and around Denver in upcoming newsletter articles. Make a week of it and see all that Colorado has to offer! The hotels have a limited number of rooms available on the weekend before the conference begins and on the Sunday evening after it ends.

AEA members Kelly Hannum and Jennifer Martineau have co-authored their third book together. Evaluating the Impact of Leadership Development is published by Pfeiffer Publishing.

From the publisher's website:
Evaluating the Impact of Leadership Development is a step-by-step guidebook for creating and implementing evaluation of leadership development systems. Approaching issues from an evaluative perspective enables leadership development professionals to consider multiple perspectives and draw lessons as a natural part of the way work is done.

Hannum and Martineau have worked together for the past 10 years. Hannum is an enterprise associate at the Center for Creative Leadership (CCL). Her work focuses on evaluation and leadership development across different cultures. Martineau is director of CCL's Design and Evaluation Center. Her work focuses on the evaluation of leadership development programs and initiatives. CCL is a top-ranked, global provider of executive education that develops better leaders through its exclusive focus on leadership education and research. Hannum and Martineau joined CCL in 1993.

"Evaluation is often still thought of as an external process, something that can be dealt with AFTER a program has taken place," says Hannum. "We wanted to provide a perspective on evaluation that highlighted it as a positive and ongoing process -- for clarifying purposes, articulating different perspectives, and learning how to effectively and efficiently reach shared, or at least agreed upon, goals."

Hannum served as chair of AEA's Business and Industry Topical Interest Group from 2005-2007 and Martineau is a member and former chair of AEA's Awards Committee.

"Leadership development evaluation is a complex undertaking and there is still much to learn," says Martineau, "but it was nice to have a project that prompts us to reflect and to document what we currently know in a way that is hopefully of value to others."

The Arkansas Group of Evaluators (AGES), AEA's newest local affiliate, has been meeting and providing training opportunities since 2005. Designed to build evaluation capacity throughout the state by sharing evaluation information and tools, AGES emerged from the tobacco prevention work sponsored by the Minority Initiative Sub-Recipient Grant at the University of Arkansas at Pine Bluff. In the process of collecting tobacco prevention data, it became apparent there was a need to build evaluation capacity throughout the state. AGES is now formally linked to the new Arkansas Evaluation Center and the evaluation certificate program at the University of Arkansas at Pine Bluff.

AGES provides an educational complement to these programs, offering specific training opportunities and is a vehicle to help with socialization of new evaluators to the field. AGES has hosted evaluation training workshops in empowerment evaluation, experimental design, quantitative methods, qualitative methods, focus groups, and interviews. In addition, educational technology workshops have been provided with a focus on online surveys, blogs, web pages, videoconferencing on the net, and telephony. AGES and the Arkansas Evaluation Center are sponsoring a free summer workshop on July 10-11.

Congratulations!

Ray C. Rist, Advisor to the World Bank, has been elected to a three year term (2008-2011) as President of the International Development Evaluation Association (IDEAS.) IDEAS is the international professional association for those working in and contributing to development evaluation. IDEAS has approximately 700 members in 95 countries. IDEAS has a ten member board with six members from developing countries and four from developed countries. Functioning largely as a virtual community, IDEAS emphasizes learning and sharing via technology. IDEAS was organized in Beijing, China in 2002 and held its first global assembly in New Delhi in 2004. Subsequently, it has worked with a number of other evaluation associations to co-host meetings and conferences. Its two most recent events were to co- host the Malaysian Evaluation Society meeting in March 2008 and the Global Environment Fund meeting in Alexandria, Egypt in May 2008.