What a wonderfully busy time of year in the
AEA office! The Internal
Scan results are ready for distribution and I
you to dive in and see what we've learned
membership - and to stay tuned as we share more
about how we're acting on those findings.
month, we had the opportunity to build
with colleagues in Washington at the meeting
Washington Research Evaluation Network. And next
week, AEA co-hosts the AEA/CDC Summer Evaluation
Institute in Atlanta. In the meantime, the
this Friday and Saturday in Denver. I am
share with them the beauty of the conference
look forward to sharing with you more about the
Board's work over the summer.
We're also looking forward to November. The
conference hotels have opened their room blocks
and we're putting the final touches on the
schedule before proposal status notices go
first week in July. The program promises to be
exciting. Plan ahead to explore the conference
program when it goes online July 3. And,
your hotel early if you wish to get into any
of the three
currently contracted blocks. With record
expected in Denver, we are likely to fill the primary
hotels well in advance of the conference.
I wish you the very best of summer.
View findings online
The results are in and ready for viewing! The American
Evaluation Association undertook an Internal Scan to
learn more about its membership during the period
September 2007 to January 2008. AEA contracted with
Goodman Research Group, Inc. (GRG) to complete
the scan and data collection. GRG surveyed the full
membership, conducted follow-up interviews and
online Question and Answer (Q&A) groups, and
compiled and analyzed the results. The results from
this scan are being used to guide strategic and
operational decision making throughout the
We are now working to disseminate the results from
that scan in a number of ways. On the scan webpage,
you can find a comprehensive report, a PowerPoint
presentation, qualitative and quantitative indexes of
analyses, and the instruments themselves.
Our analysis and dissemination work is not complete.
The first open-ended question of the survey generated
hundreds of pages of qualitative responses. We are
diving into that rich content further, as well as
conducting additional analyses based on requests
from the ten AEA standing committees currently
reviewing the scan.
At the upcoming conference, we will offer three
separate sessions focusing on the scan.
- 1. The first will focus on the basic findings
and the ways in which AEA is acting on them.
- 2. The second will focus on the process and
methodology used in conducting the scan.
- 3. The third will focus on the extensive data
from the remaining open-ended question, elaborating
on what it means to be an evaluator.
Over the coming months, we will use the newsletter to
share findings of particular interest, as well as to keep
you informed about the related sessions at the
conference and next steps based on the scan's
While the documents shared online include an
extensive set of analyses, suggestions for other
analyses of the Internal Scan data may be sent to
Susan Kistler in the AEA office at email@example.com. If
you have any questions, please do not hesitate to
forward to the same address.
Go to the Internal Scan webpage to download reports and instruments
Meet Rick Davies from M&E News
As part of AEA's commitment to help keep
members aware of global developments in
we are launching a series of interviews with notable
figures within the international evaluation community.
Our first interview is with Rick Davies,
whose "Monitoring and Evaluation News" website
http://www.mande.co.uk) and its associated email
list connect evaluators in far-flung countries and offer
links to a wealth of evaluation resources. An
independent consultant based in the United Kingdom
(UK), Rick also developed the now widely-used tool
known as "Most Significant Change" (MSC). Below are
some excerpts from the interview:
"The email list now has more than 1,700 members
worldwide, which makes it one of the largest in the
field. It also has a noticeably high proportion of
members from Africa and Asia, which is great to see...
"Overall, I continue to be surprised how helpful people
on the email list can be to others, especially when
some don't seem to have done much
prior 'homework!' The more interesting and longer
term challenge is to identify where the email list and
website best fit within an increasingly large and
complex ecology of other websites and email lists
dealing with monitoring and evaluation issues..."
"[One development that concerns me] has been the
increasing proportion of bilateral aid that is being
spent via Direct Budget Support (DBS) to governments
that are believed to have some minimal level of
competence... My experience with the Ghana Poverty
Reduction Strategy has highlighted how limited
government capacities can be, how hard it can be to
develop that capacity, and how limited donors'
interests are in a critical review of that capacity. For
many international donors, the use of DBS is
administratively convenient and ideologically
comfortable. This limits the type of evaluation findings
that are likely to be listened to..."
"Lack of transparency can be an indicator of
ineffectiveness, in that those responsible for corrupt
and / or ineffective practices are not usually keen for
these practices to be made publicly visible. On the
other hand, the presence of transparency can be an
instrument for increasing effectiveness, because, as
they say in the world of software design, 'with enough
eyeballs, all bugs [problems] are shallow...'"
"...(A)id projects must not only have an impact and be
able to identify that impact. They must also be able to
communicate to others how to replicate that impact."
For more information about Rick and his
perspectives on evaluation, the complete interview is
available on the AEA website. Thanks to members of
AEA's International Committee for their assistance
with this interview.
Go to full online interview
Connecting with research evaluators
Evaluation is always a topic of great interest in the
nation's capital, and AEA is increasingly an important
part of the discussion. The association worked with
the Washington Research Evaluation Network
(WREN) to support and learn from a June 6
conference at George Washington University that
brought evaluators together with policymakers,
program managers, and evaluation managers from
science and technology organizations.
AEA President Bill Trochim delivered the keynote
address, in which he described the work of the
association's Evaluation Policy Task Force that is
encouraging federal agencies to incorporate sound
evaluation policies in their efforts to increase
accountability. He also took part in a discussion panel
at the day-long conference.
"This conference was a bit of an experiment around
AEA's engagement in the Washington area," Trochim
said. "Working with AEA members associated with the
federal government, we hope that we might be able to
develop a variety of mechanisms that will enhance our
ability to influence federal evaluation policies."
Gretchen Jordan, chairman of AEA's Research,
Technology, and Development Evaluation TIG, said
her group and WREN had been working jointly in this
area for the last several years - with the TIG holding
academic/practitioner-oriented meetings and WREN
hosting international workshops for government policy
makers and program managers.
Research in this area is now being funded by the
National Science Foundation's "Science of Science
and Innovation Policy" (SciSIP) initiative, and their
announcements describe the sorts of methodology
development for which AEA members can apply for
funding. New methodologies developed by SciSIP will
be presented for discussion at the next WREN
meeting in December this year.
Evaluation is also receiving substantial attention from
a White House Office of Science and Technology
Policy (OSTP) initiative on the "Science of Science
Policy. An OSTP Science of Science Policy task force
comprising evaluation representatives from federal
R&D agencies is developing a "road map" of
research, data, and data tools needed to develop this
science. A preliminary draft of the road map is
expected sometime this summer and will be a major
topic at the next WREN meeting.
"We're still in the beginning phases of coming up with
the best tools for evaluating R&D," said Cheryl Oros,
coordinator of the conference and a member of
WREN's Steering Committee as well as the OSTP
Science of Science Policy workgroup. "It's important
that we bring everybody together to see if anybody in
federal agencies can allocate resources for this and
sponsor pilot evaluations using new tools, perhaps
teaming on efforts. How do we talk the policymakers
into investing in these tools? We see AEA as an
important part of this process."
Go to the WREN Wiki for the meeting proceedings
Room blocks now available
The conference room blocks are now open for
Evaluation 2008! Although the conference seems to
be a long way off, the hotel room blocks will fill well in
advance of the event. If you will be attending the
conference, and wish to stay at your first choice hotel,
please make reservations at your earliest
convenience. Rooms will be filled on a first-come,
first-served basis and rooms outside of the blocks are
at considerably higher prices.
We hold blocks at three hotels, each with its own
- The Hyatt Regency is the headquarters
hotel for the event. New and modern, this four
diamond hotel offers outstanding service right in the
heart of the event. Approximately 2/3 of the conference
sessions will be held here with the other 1/3 across
the street at the convention center. Rates at the Hyatt
start at $159.
- The Hilton Garden Inn is next door to the
Hyatt and is a great affordable option with a 24 hour
complimentary business center. The rooms are
freshly renovated, and complimentary internet rounds
out the package. Rates start at $129.
- The Magnolia Hotel is a modern boutique
property two blocks from the Hyatt with complimentary
internet, cookies, and breakfast! The updated rooms
and soaking tubs, and comfy public spaces make it a
fun alternative. Rates start at $140.
The pre-conference workshops run from Monday,
November 3, through 3:00 pm on Wednesday,
November 5. The conference itself begins on
Wednesday at 3:15 and runs through Saturday,
November 8, at 5:30 pm. Post-conference workshops
round out the event on Sunday, November 9, from 9:00
am to 12:00 pm. We'll be telling you more about the
wonderful things to see in and around Denver in
upcoming newsletter articles. Make a week of it and
see all that Colorado has to offer! The hotels have a
limited number of rooms available on the weekend
before the conference begins and on the Sunday
evening after it ends.
Take a moment to review the hotel descriptions and
insider tips online at
There you will also find details on how to book into the
discounted conference blocks.
Impact evaluation the CCL way
AEA members Kelly Hannum and Jennifer Martineau
have co-authored their third book together.
Impact of Leadership Development is published
by Pfeiffer Publishing.
From the publisher's
Impact of Leadership Development is a
step-by-step guidebook for creating and implementing
evaluation of leadership development systems.
Approaching issues from an evaluative perspective
enables leadership development professionals to
consider multiple perspectives and draw lessons as a
natural part of the way work is done.
Hannum and Martineau have worked together for the
past 10 years. Hannum is an enterprise associate at
the Center for Creative Leadership (CCL). Her work
focuses on evaluation and leadership development
across different cultures. Martineau is director of
CCL's Design and Evaluation Center. Her work
focuses on the evaluation of leadership development
programs and initiatives. CCL is a top-ranked, global
provider of executive education that develops better
leaders through its exclusive focus on leadership
education and research. Hannum and Martineau
joined CCL in 1993.
"Evaluation is often still thought of as an external
process, something that can be dealt with AFTER a
program has taken place," says Hannum. "We wanted
to provide a perspective on evaluation that highlighted
it as a positive and ongoing process -- for clarifying
purposes, articulating different perspectives, and
learning how to effectively and efficiently reach shared,
or at least agreed upon, goals."
Hannum served as chair of AEA's Business and
Industry Topical Interest Group from 2005-2007 and
Martineau is a member and former chair of AEA's
"Leadership development evaluation is a
complex undertaking and there is still much to learn,"
says Martineau, "but it was nice to have a project that
prompts us to reflect and to document what we
currently know in a way that is hopefully of value to
Go to CCL's Online Bookstore
Arkansas group marks AEA's 25th
The Arkansas Group of Evaluators (AGES), AEA's
newest local affiliate, has been meeting and providing
training opportunities since 2005. Designed to build
evaluation capacity throughout the state by sharing
evaluation information and tools, AGES emerged from
the tobacco prevention work sponsored by the Minority
Initiative Sub-Recipient Grant at the University of
Arkansas at Pine Bluff. In the process of collecting
tobacco prevention data, it became apparent there
was a need to build evaluation capacity throughout the
state. AGES is now formally linked to the new
Arkansas Evaluation Center and the evaluation
certificate program at the University of Arkansas at
AGES provides an educational complement to these
programs, offering specific training opportunities and
is a vehicle to help with socialization of new evaluators
to the field. AGES has hosted evaluation training
workshops in empowerment evaluation, experimental
design, quantitative methods, qualitative methods,
focus groups, and interviews. In addition, educational
technology workshops have been provided with a
focus on online surveys, blogs, web pages,
videoconferencing on the net, and telephony. AGES
and the Arkansas Evaluation Center are sponsoring a
free summer workshop on July 10-11.
The group is AEA's 25th local affiliate and was
organized by its chair, Linda Delaney and David
Fetterman, board member and past president of AEA.
The group operates on a calendar year with elections
held in January. Officers are in place for a three-year
term and terms are renewable. For more information,
contact the chair at
Go to the Arkansas Group of Evaluators Blog Spot
Changes in the evaluation community
Ray C. Rist, Advisor to the World Bank, has
been elected to a three year term
(2008-2011) as President of the International
Development Evaluation Association
(IDEAS.) IDEAS is the international professional
association for those working in and contributing to
development evaluation. IDEAS has approximately
700 members in 95 countries. IDEAS has a ten
member board with six members from developing
countries and four from developed countries.
Functioning largely as a virtual community, IDEAS
emphasizes learning and sharing via technology.
IDEAS was organized in Beijing, China in 2002 and
held its first global assembly in New Delhi in 2004.
Subsequently, it has worked with a number of other
evaluation associations to co-host meetings and
conferences. Its two most recent events were to co-
host the Malaysian Evaluation Society meeting in
March 2008 and the Global Environment Fund
meeting in Alexandria, Egypt in May 2008.
2007 AEA Award Winners
The names of
the 2007 AEA award winners are now online and
permanently archived via an article recently published
in the American Journal of Evaluation, Volume
29 Issue 2. Congrats
again to Arlen Gullickson, Karen Kirkhart
and Liliana Rodriguez-Campos. As an
AEA member, you have free access to all current and
back content from AJE. Just sign on using your
username and password to the members only
section of the website at http://www.eval.org/.