|
|
| Newsletter: October 2008 |
Vol 8, Issue 10 |
|
|
|
|
|
| |
AEA Colleagues,
AEA needs your input, thoughts, and feedback. At this time, we are lucky to enjoy a strong financial footing and our Board is excited about moving into a new era of policy governance. This will shift how we do business - but retain and even strengthen our core commitment to the values, programs, and people who have made this a vibrant, valued, and welcoming community of practice. We want to be sure that we remain true to ourselves, yet can manage our considerable growth. That we take care of the present, but prepare for the future in ways that will make AEA ten years from now an even stronger professional home for the next generation of members - and for those of us who hope to be practicing still. One of the fundamental tenets of the AEA Board's approach to policy governance is more actively and continuously listening to, and learning from, the members of the association. Earlier this year we shared with you the data from the member scan (for more information go to http://www.eval.org/Scan/aea08.scan.htm), and now we have two other key items for your consideration. Please take a moment to review these two documents, and provide feedback, raise critical questions, or identify any concerns you may have: Document 1 - The AEA Strategic Framework: The Framework lays out AEA's Vision, Mission, Values, and Goals. The Board of Directors has been working on this and has reflected on and incorporated input from the over 50 volunteer members comprising the association's 11 standing committees. You may have seen a version of the values statement in last year's conference program or had a discussion with one of the association's leaders about our strategic planning efforts. Please go to http://www.eval.org/aea08.framework.asp, review the Framework, and provide any feedback, via the form provided there, that you feel would be valuable for future drafts or for us to consider as we work to make real all that is contained within that document. Does the AEA Strategic Framework reflect what you believe are our principal priorities and commitment? Document 2 - The Building Diversity Initiative Phase I Evaluation Report: AEA contracted with Two Gems Consulting to conduct an evaluation of the 14 recommendations emerging out of the Building Diversity Initiative, begun almost 8 years ago. This first phase of the BDI evaluation was an opportunity to take stock of the status of each recommendation, to gain a richer understanding of where the association had taken action, where it had not, and whether we should continue on some paths or start anew with others. The report suggests that we consider what it would be like to be a truly multicultural association and documents a part of our history in this important area.
Please go to http://www.eval.org/aea08.bdi.eval.asp, and review the Phase I BDI Evaluation. I'd like you to view it not only as an evaluation document, but also as an opportunity to learn about the BDI itself if you are not familiar with it. Then please take some time to provide any feedback, via the form provided there, that you believe is important as we move forward with the next chapter in our efforts to make AEA a genuinely diverse, inclusive, and welcoming organization that values all of our members and their contributions to the field. I appreciate your willingness to share you thoughts. Please read and respond by Friday, October 24, 2008. Thank you for your time, your input, and your investment in the association,  William Trochim 2008 President, American Evaluation Association |
|
 |
|
|
|
| 08 AEA Award Winners Announced |
|
AEA's annual awards this year will recognize two newcomers, a veteran, a scholar and a collective group of 16 authors whose ground-breaking work has been used to more effectively evaluate community-based prevention programs worldwide. AEA's awards - honoring people or groups who've contriubted greatly to the field of evaluation - will be presented at Evaluation 2008, to be held Nov. 3-8 in Denver, Colorado. The awards presentation is scheduled for Friday's luncheon and will be followed by Conversation Hour with the 2008 Award Winners, a unique Q&A session offered by AEA's Awads Committee that provides an opportunity for AEA members to meet and interact with the national award winners. This year's winners are:
The 2008 Alva and Gunnar Myrdal Government Award, Stephanie Shipman, United States Government Accountability Office, Washington, District of Columbia. Presented to individuals whose evaluation work is highly influential in governmental contexts and who have been particularly instrumental in furthering the interests of evaluation though advocacy, sponsorship, management, or use of evaluation in government.
The 2008 Paul F. Lazarsfeld Award, J. Bradley Cousins, University of Ottawa, Ottawa, Ontario, Canada. Presented to an individual whose written work on evaluation theory has led to fruitful debates on the assumptions, goals, and practices of evaluation.
The 2008 Marcia Guttentag Award, Kelly Hannum, Center for Creative Leadership, Greensboro, North Carolina; and Chris L. S. Coryn, Western Michigan University, Kalamazoo, Michigan. Presented to a promising new evaluator during the first five years after completion of his or her Masters or Doctoral degree and whose work is consistent with the AEA Guiding Principles for Evaluators.
The 2008 Outstanding Publication Award, Getting To OutcomesTM (GTO), the RAND Corporation, Santa Monica, California. Presented for a publication that has been instrumental to the development of theory or practice in the field of evaluation. The GTO publications were written by Matthew Chinman, Pamela Imm and Abraham Wandersman and 13 others associated with the RAND Corporation, the Centers for Disease Control and Prevention, Join Together, University of South Carolina, and the New York State Office of Child and Family Services, Search Institute. Chinman is an employee of the RAND Corporation, Imm and Wandersman are affiliated with the University of South Carolina.
|
| Interview with Robert Shea, OMB |
 In the last newsletter, we shared with you the comments that AEA's Evaluation Policy Task Force (EPTF) provided to Robert Shea at the United States Office of Management and Budget (OMB) on the role of evaluation in OMB's Program Assessment Rating Tool (PART) program. This month, we'd like to share some excerpts from a recent interview with Shea, who directs the Bush administration's Performance Improvement initiative at OMB.
AEA's engagement with OMB has extended well beyond that initial outreach. We have established an ongoing exchange of ideas during a period when OMB and senior government evaluators and program managers are reassessing their program performance assessment protocols. The EPTF chair was the featured speaker at the first meeting of the Program Evaluation Working Group of the Performance Improvement Council established through OMB. This group is made up of senior representatives of federal agencies and is charged with ongoing improvement in evaluation processes and policies. EPTF consultant George Grob also maintains active contact with senior staff involved in OMB's new initiative to review and improve the PART program. Recently, Grob and AEA President/EPTF task force chair William Trochim had another opportunity to talk with Shea about evaluation and the role AEA can play to provide expert advice on national evaluation policies.
During our discussions, Shea spoke of a deep commitment to effective government, noting that he has learned a considerable amount about the nuances of evaluation through leading the PART effort, and hoping that an institutional acceptance of evaluation becomes more of an integral part of governance. He acknowledged the importance of outcomes measurement and the challenges in attributing outcomes directly to a program:
"...Some programs still don't have a clear enough idea how to measure success. Furthermore, they don't know whether what they're doing, as distinguished from some other factor, is what's causing performance to improve or deteriorate."
Shea made a point of noting the important qualifications in PART policy regarding the role of randomized controlled trials:
"...Over the last several years, we have begun to pay more attention to the methodologies employed in program evaluations. PART guidance, however, clearly states that there is no requirement to perform a Randomized Control Trial or any other specific methodology. Indeed, the guidance states, "Overall, evaluations must be appropriate to the type and size of the program. Agencies and OMB should consult evaluation experts, in-house and/or external, as appropriate, when choosing or vetting rigorous evaluations."
And, he described the critically important role that professional evaluators can play in shaping federal program improvement efforts:
"...Programs need help designing cost-effective evaluations to answer questions about program impact and management. Helping program managers design, implement, and interpret evaluations is the greatest contribution professional evaluators can make to the Federal government's program improvement efforts."
Even before coming to the White House, Shea worked as a key legislative staffer in both the House and Senate on initiatives to assess the effectiveness of federal programs. He will soon be moving from his position in the federal government to one in the private sector, but he will almost certainly continue to be an important voice in the shaping of management and evaluation at the federal level.
Go to the full online interview |
| AJE Ranked Among Top Social Sciences Journals |
 The American Journal of Evaluation (AJE) was recently ranked among the top interdisciplinary social sciences journals in terms of impact on the field. In Thomson Reuters Journal Citation Reports' annual ranking of academic journals, AJE had an impact factor of 1.298 in 2007, ranking sixth among the 57 publications that were reviewed in its category.
"This above-average impact factor is good news for AJE, since librarians consider a journal's impact factor as an important criteria for keeping or cancelling a subscription," says AJE Editor Robin Miller. "But it's also good news for authors of AJE articles, because it shows they are having a strong impact on the field."
Each year, Thomson Reuters indexes roughly 7,500 journals from more than 3,300 publishers from 60 countries, divides them into subject categories and calculates each journal's "impact factor." A journal's impact factor is the average number of times that journals published in one year cite articles that are published in the preceding two years.
For the 2007 Journal Citation Report�, the impact factor was calculated by dividing the number of 2005-2006 AJE articles cited in 2007 by the total number of AJE articles published in 2005-2006. In the social sciences category, anything over 1.000 is considered high-impact.
AJE articles were cited a total of 248 times in AJE and other publications during 2007. AJE articles were cited in more than 100 publications in fields ranging from preventive medicine and substance abuse to environmental policy and public administration.
Miller said AJE and its publisher, SAGE Publications, have consciously worked to maintain a high impact factor in a number of ways.
"We've taken steps to raise AJE's visibility, like improving our online presence and pre-publishing submissions online before they appear in print," she said. "We've also increasingly internationalized the publication and expanded the diversity of authors from around the world. But you can't overlook the importance of high-quality material, and we try to solicit the best in the field."
At the Evaluation 2008 conference in Denver, Miller will be taking part in a workshop to provide guidance for first-time authors on how to publish an article in AJE. Demonstration Session 684 will be held from 4:30 PM to 6:00 PM on Friday, November 7. A similar session last year proved to be very popular, attracting more than 100 participants. To view publication guidelines, visit http://www.eval.org/Publications/AJE.asp. |
| AEA Leadership & The Road to Success |
The Nominations & Elections (N&E) Committee for the second year is offering attendees of AEA's annual Evaluation conference the opportunity to learn more, from a first-hand vantage point, of what it takes to get elected to key leadership positions. Leadership Recruitment and the Road to the American Evaluation Association Board of Directors will be held Thursday, November 6, at 3:35 p.m. in the Capitol Ballroom Section 4 in the Hyatt.
"As the current chair of N&E I have seen a lot of interest among members about serving the organization. Unfortunately, there is no clear path of involvement," says 2008 N&E Chair Sarita Davis. "But be encouraged, AEA will soon be offering new and creative ways to become involved in the organization."
Think Tank Session 343 is targeted toward those interested in taking on a leadership role within AEA with feedback from those who've served in that capacity. Discussants include incoming President Debra Rog and 1994 President Karen Kirkhart, as well as AEA board members past and present including Rakesh Mohan, Stan Capela and Sarita Davis. Some found themselves in leadership roles almost effortlessly; others made several unsuccessful attempts before achieving their goals. They, and others, will share tips for success as well as lessons learned. The session will also include an introduction of members of AEA's Leadership Task Force, who will be on hand to answer questions and coordinate follow-up with all who express interest in leadership opportunities.
From the meeting abstract:
"An organization's strength and viability is dependent on strong leadership. A major component of such leadership resides with the AEA Board and its commitment to identify future leaders as well as foster a culture that encourages member participation to seek nomination to the Board. One of the major drawbacks is often a lack of knowledge on the part of members on what are the key ingredients to becoming a viable candidate for the Board. Further, there are those members who seek nomination and often fail to either be selected for the slate or who run for the Board but do not win. The purpose of this session is to provide several case studies of individuals who have participated on the Board over the past few years. They will provide an overview of their experience as well as the highs and lows of seeking nomination to the Board. Further, the session will help educate members on the role of the nominations and elections committee as well as the leadership task force to assist members who may seek to run for the board and/or who have lost and want additional information on steps that can be taken to increase the likelihood of winning a slot on the board. The overall purpose of this session is to help the organization identify future leaders but more importantly encourage greater member participation by seeking nomination to the board and further strengthening organization capacity."
|
| Diversity Committee Pays Tribute to Educator Asa G. Hilliard |
 AEA's Diversity Committee will explore the life and legacy of Dr. Asa Grant Hilliard III, world renowned Pan-Africanist educator, historian, and psychologist, at this year's Evaluation 2008 - and pose to participants the probing questions for which he was known. Hilliard died last year at the age of 73 in Cairo, where he was scheduled to deliver a keynote lecture at the annual conference of the Association for the Study of Classical African Civilization. Though not an evaluator, Hilliard championed the confluence of cultures and emulated a practice of thoughtful debate aimed at raising consciousness of cultural issues.
Hilliard's work took him from Denver and Atlanta to Egypt and Africa. For 30 years, he led study groups to Egypt and Ghana where he taught the history of Africa and the African Diaspora. He was a prolific writer and was co-developer of an educational television series entitled Free Your Mind, Return to the Source: African Origins. Hilliard served on the faculty of San Francisco State University for 18 years during which he was a department chair, dean of education and a consultant to the Peace Corps and Superintendent of Schools in Monrovia, Liberia. He consulted with many of the leading school districts, universities, government agencies, and private corporations on valid assessment, curriculum equity and teacher training. Several of his programs in pluralistic curriculum, assessment, and valid teaching have become national models.
Hilliard was born in Texas and graduated from Manual High School (1951) in Denver, Colorado. He received a B.A. from the University of Denver (1955) and taught in the Denver Public Schools before joining the U.S. Army. He later received his M.A. in Counseling (1961) and Ed.D. in Educational Psychology (1963) from the University of Denver. In pursuit of his education, Hilliard worked in many occupations including as a teacher, railroad maintenance worker, bartender, waiter and cook. This celebration is very special in that Hilliard's legacy is being honored in the city in which his career began.
The Diversity Committee's think tank, co-sponsored by AEA's Indigenous Peoples in Evaluation, Multiethnic Issues in Evaluation, and Social Work TIGS, is designed to orient participants to Hilliard's mindful consideration of cultural context. Small group discussions will focus on 1) the cultural ethos in the design, methodology and interpretation of results in the context of evaluation; 2) creating common language about the work to connect theory and practice; 3) the value of social transmission and how we define what is valuable and worth knowing; and 4) recognizing the importance of questions being asked and the paradigm from which they originate with a shift from deficit inquiry (what is wrong or lacking) to appreciative inquiry (starting with successes).
Think Tank Session 390 is scheduled for Thursday, November 6, 4:30-6 PM, in the Capitol Ballroom Section 5. For more information, visit AEA's Conference Program. To learn more about Hilliard, visit http://www.asaghilliard.net/. |
| AEA Partners with "Denver's Road Home" to Help Feed the Homeless |
 AEA has long advocated using evaluation work to help bring about social betterment. For this year's Evaluation 2008 conference, we're expanding that concept to give attendees a chance to promote social betterment by actually providing change - the spare change in their pockets!
While in Denver, AEA is partnering with "Denver's Road Home," Mayor John Hickenlooper's 10-year plan to end homelessness in the city. Part of that effort is the Donation Meter Program, where specially marked red parking meters are installed across town to create community awareness about Denver's Road Home and discourage panhandling by providing a place to donate change to fund local initiatives that provide meals, job training, substance abuse counseling, housing, and other programs designed to assist the homeless in living life off of the street.
During Evaluation 2008, one of those meters will be set up in the foyer outside the main conference ballroom in the Hyatt Regency Hotel. We encourage everyone to bring lots of change to feed this meter, as well as other meters located around town.
"We appreciate the American Evaluation Association's support of our efforts to provide an innovative approach to ending homelessness in Denver," said Hickenlooper. "This project has been successful in educating the community that panhandling and homelessness are not synonymous and that, if you want to help homeless people, there is a better way to give."
A survey conducted by the Downtown Denver Partnership suggested that Denver residents were giving approximately $4.5 million annually to panhandlers. In the program's first year, city officials say nearly $100,000 was collected, while panhandling in the downtown area was reduced by 92 percent. To better understand the impact of Denver's Road Home, local AEA members at OMNI Institute were selected to evaluate the overall initiative.
"When the mayor adopted the issue of homelessness, accountability and evaluation emerged as a priority," said Katie Page, OMNI's evaluation lead for the Denver's Road Home project. "Evaluation has influenced policies and practices at every level, from government contracting to the use of data at local non-profits."
At Evaluation 2008, Ms. Page will take part in a session with representatives from the Denver's Road Home project management team and Denver C.A.R.E.S. - a local addiction treatment and detoxification center funded through "Denver's Road Home". The panel will discuss Denver's Road Home and the approach this initiative has taken to evaluation through the three perspectives of the panelists. Panel Session 709 will be held from 4:30 p.m. to 6:00 p.m. on Friday, Nov. 7, in Room 104 in the Convention Center.
|
| The Logic Model Guidebook |
 AEA members Lisa Wyatt Knowlton and Cynthia C. Phillips have authored a new book published by SAGE. The Logic Model Guidebook: Better Strategies for Great Results, released in October by SAGE Publications, serves as a resource guide for practitioners and consumers.
From the Publisher's Website:
"The Logic Model Guidebook offers a concise, practical overview of the logic modeling process as applied to numerous organizational contexts. Authors Lisa Wyatt Knowlton (Ed.D.) and Cynthia C. Phillips (Ph.D.) examine the structures, processes, and language of logic models as an emerging tool that improves the design, development, and implementation of change efforts within programs and greater organizational initiatives. Through concise, step-by-step process articulation, enhanced by numerous visual learning guides (sample models, checklists, exercises, worksheets) and case examples, the authors provide students, practitioners, and beginning researchers with invaluable tools to develop and improve these models."
From the Authors:
"We make logic models very concrete though the use of real world examples and the concept of archetypes. Archetypes streamline and simplify some of the more time-consuming aspects of creating or using models. We wrote the Guidebook for people, like us, who want to get results. We realized after reflecting on numerous conversations we have had with colleagues and clients that a "pretty picture" was not enough to secure success."
About the Authors:
Phillips has been an active member of AEA since 1995. She is co-chair of the Cluster, Multisite, Multilevel TIG and has been involved with the Mid Career Evaluator Task Force, which was instrumental in bringing Learning Circles to AEA as a new professional development opportunity for members. Wyatt Knowlton has been a member of AEA for 6 years, is a member of the non-profit TIG and her interests are strategy and performance management.
AEA members receive a 20 percent discount on books from SAGE when ordered directly from the publisher. The discount code for AEA members is SO5CAES or members can call the Customer Care department at 1-800-818-7243.
Go to the Publisher's Website |
| The Handbook of Social Research Ethics |
 AEA members Donna M. Mertens and Pauline E. Ginsberg have collaborated to produce a new book that was released in August by SAGE Publications. The Handbook of Social Research Ethics addresses ethical issues that arise in the theory and practice of research within a technologically advancing and culturally complex world.
From the Publisher:
"The Handbook of Social Research Ethics is the first comprehensive volume of its kind to offer a deeper understanding of the history, theory, philosophy, and implementation of applied social research ethics. Editors Donna M. Mertens and Pauline Ginsberg bring together eminent, international scholars across the social and behavioral sciences and education to address the ethical issues that arise in the theory and practice of research within the technologically advancing and culturally complex world in which we live. In addition, this volume examines the ethical dilemmas that arise in the relationship between research practice and social justice issues."
From the Editors:
"Members of AEA will see many familiar names among the contributors (e.g., Robert Stake, Mel Mark, Yvonna Lincoln, Hazel Symonette, Veronica Thomas, Joan LaFrance, and Fiona Cram). In addition, scholars from around the world contribute to expanding the discussion of ethics in applied social research and program evaluation. What do advances in technology mean for ethical conduct of inquiry? What are the implications of globalization for ethics? How can future researchers and evaluators be trained to be ethical in their practice? These are some of the issues dealt with in this 37-chapter volume."
About the Editors:
Mertens is a professor in the Department of Educational Foundations and Research at Gallaudet University. She is a former AEA President and board member. Ginsberg is Professor of Psychology at Utica College. She is a founding member and past co-president of AEA's International and Cross-Cultural Evaluation Topical Interest Group and past president of the Eastern Evaluation Research Society.
AEA members receive a 20 percent discount on books from SAGE when ordered directly from the publisher. The discount code for AEA members is SO5CAES or members can call the Customer Care department at 1-800-818-7243.
|
| AZENet Conference |
The Arizona Evaluation Network (AZENet) will hold its fall conference Strengths-Based Evaluation & Consulting on Tuesday, November 18. Join other evaluators and consultants in an active, discussion-oriented look at recent trends and learn about strengths-based perspectives that will help improve your practice. This AEA affiliate sponsored event includes lunch and will be held at the Franciscan Renewal Center at 5802 E. Lincoln Dr., Scottsdale, Arizona.
AZENet is an organization for Arizona professionals involved with or interested in program evaluation. It was founded in 1997 to support the broad and diverse group of evaluators in Arizona and its mission is to provide a statewide interdisciplinary forum for professional development, advocacy, networking, and exchange of theoretical, methodological, and practical knowledge in the field of evaluation. AZENet members represent private consulting firms, government agencies and universities.
The conference costs $25 for members; $30 for non-members; and $15 for full-time students. For more information, visit http://www.azenet.org/Conferences_and_Clusters.html. |
| AES Expands Online Features |
The Australasian Evaluation Society (AES) recently introduced two online features that may be of interest to AEA members. Stories About Evaluation Learnings (SEL) aims to share current experiences, critical perspectives and methodological innovations among practitioners of evaluation while Evaluation Reports is intended to share examples of reporting styles. "While there are many opportunities to access scholarly articles in academic journals, there has been little systematic access to working evaluation reports," notes John Owen, chair of AES' Publications Committee. "AES welcome contributions to both new categories. In fact, your input will be essential if these sections are to make an ongoing contribution to the dissemination of evaluation practice and theory in Australasia. With SEL, we aim to keep the series informal to enable the rapid sharing of practical experience from the field and reflections on that practice." Guidelines for submissions can be found at http://www.aes.asn.au/publications/Stories/ and http://www.aes.asn.au/publications/Reports/. Also available for online viewing is AES' international journal, Evaluation and Program Planning, as well as the Evaluation Journal of Australasia. Each issue is archived with a table of contents readily available. |
| Get the Most of Your Membership |
| As fall approaches, we draw nearer to AEA's annual Evaluation conference and the fall academic year. As always, there are many ways right now to participate in the life of the association. Please click through to the appropriate item to learn more.
|
| About Us |
| The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275
| |
|
|
|
|
|
|
|
|
