|Newsletter: November 2009
||Vol 9, Issue 11|
|The Numbers Speak Volumes - Evaluation 2009|
We welcomed 2460 attendees to this year's Evaluation 2009 conference in Orlando, Florida, attendance on par with last year's 2550 in Denver. I am especially encouraged by the strength of AEA in times when many professional associations have seen declines in both membership and participation. My sincerest thanks go out to everyone who contributed. Your investment, enthusiasm, and energy were evident as we prepared for the annual conference in a multitude of ways, many of them behind-the-scenes and far from fanfare. I wanted to share some of the numbers associated with the conference and your commitment:
- 2603 Program Contributors
- 1000+ Presenters
- 743 Proposal Reviewers
- 617 Sessions
- 126 Topical Interest Group Leaders
- 112 Student Volunteers
- 54 Workshop Facilitators
- 42 Ambassadors
- 26 Minority Serving Institution Faculty
- 18 Night's Out Event Hosts
- 13 Board Members
- 9 Graduate Education Diversity Internship Participants
- 4 Pipeline Opportunities Program Students
- 2 Presidential Strand Coordinators
- 1 Local Arrangements Chair
The numbers speak volumes and I am so proud to be affiliated with an association with such active membership and participation.
Peace and joy to you and yours this holiday season,
AEA Executive Director
|Policy Watch - OMB Policy on Impact Evaluation|
From George Grob, Consultant to the Evaluation Policy Task Force
The United States
Federal Office of Management and Budget (OMB) is seeking comments on implementation of the Paperwork Reduction Act. AEA
is planning to send comments and is seeking your
input by Wednesday, December 2. Please read on to learn more and make your
On October 27, OMB published a request for comments
the requirements of the Federal Paperwork Reduction Act
Under the Act, all surveys and other data collection instruments issued by or
on behalf of any Federal agency to more than 9 responders must be approved in
advance by OMB. OMB is seeking comments on reducing current paperwork
burdens, especially on small entities; increasing the practical utility of
information collected by the Federal Government; ensuring accurate burden
estimates; and preventing unintended adverse consequences.
The review process is complex. It involves an initial review by the
originating Federal agency, two rounds of public comments (first for 60 days,
and then again for 30 days), and up to 60 days of review by OMB. The entire process usually takes at least six months and often
substantially longer than that. While addressing legitimate concerns about paperwork burdens on the public, it may
delay the availability of critically important evaluation results.
All AEA members may respond directly to OMB's request for
comments on their own. In addition, AEA plans to send a short statement to OMB
on this important issue. To inform this statement, we would like member input. We
are particularly interested in hearing from AEA members who have had experience
in seeking approval of evaluation surveys or other data collection instruments
under the Paperwork Reduction Act. The schedule for making our reply is very
tight. We need to hear from you by 5:00
PM EST on Wednesday, December 2, via our online questionnaire.
Of course, this
should not preclude your submitting your own comments directly to OMB and I encourage you to read the request for comments
to learn more regarding options for direct input.
Please feel free to share this information with your colleagues and
invite them to respond as well.
We are grateful for your assistance. Go to the EPTF website to join the discussion group and learn more about its work
|AEA Honors DOT Employee with Myrdal Government Award |
|The American Evaluation Association honored a Washington, DC-based federal employee with its 2009 Alva and Gunnar Myrdal Government Award. His work has been instrumental in safety initiatives that led to reduced derailments and fewer injuries, as well as new federal safety legislation.
Michael Coplen is a Senior Human Factors Program Manager with the U.S. Department of Transportation's (DOT) Federal Railroad Administration (FRA). He began his career some 30 years ago, first as a brakeman and then as a locomotive engineer. As a young rail employee, he became increasingly concerned about the dangers brought on by fatigue resulting from long hours, irregular scheduling, and interrupted sleep.
Being naturally inclined toward evaluation, and long before he knew the field existed, Coplen began recording his own sleep-wake schedule and reported his observations in the news media, citing bouts of up to 26 hours with no sleep and one of every 6 days with less than 5 hours of sleep. He has dedicated the last 20 years to serving as a bridge for change. Through Coplen's efforts to spearhead safety initiatives, his pilot projects in the rail industry have seen significant reductions in at-risk behaviors and locomotive engineer de-certifications, as well as improvements in rule violations and personal safety awareness.
Coplen's persistent efforts with pilot demonstration projects were an essential part of the development of FRA's Risk Reduction Program. Coplen currently manages $2 million of federal funding each year and, over the past 10 years, has been responsible for designing, implementing, and evaluating numerous pilot demonstration projects - the majority of which are related to safety culture change. Overall, these pilot programs have resulted in significant safety improvements, including: an 80% drop in injury rates and 76% drop in reportable injuries in a station services department, a 72% drop in locomotive engineer de-certification rates with road crews, a 69% reduction in human factors yard derailment rates, an 86% drop in rules violations, a threefold increase in car moves between derailments, and marked improvements in labor management relations.
"Mike is committed to conducting research and evaluations that have an impact on safety," says Michael Quinn Patton, who nominated Coplen for the AEA award. "He's learned that getting findings used is complex. He says, "If I've learned anything from my experience as a program manager over the past 10 years it's that all too often our evaluation research reports do little more than gather dust in the hallowed halls of federal science. I've come to realize that there is an art, a craft, and a science to increasing the use and impact of R&D projects and evaluation." He has become an expert in the art, craft, and science of government evaluation."
Other recipients of AEA's 2009 awards - presented at Evaluation 2009 - will be profiled in future issues.
|TechTalk - Revisit Evaluation 2009 with AEA's Online Resource Library |
|From LaMarcus Bolton, AEA Technology Director
This may come as no surprise to any of the attendees, but our Evaluation 2009
conference was an overall success. However, despite all the wonderful and informative workshops and sessions offered, I'm sure that schedule conflicts prevented many from attending certain sessions of interest. Further, for those of you who did sessions of interest, wouldn't you love to revisit slides and other session materials post-conference? Well, AEA's resource library was created to serve as a knowledge management system for evaluation-relevant information, including conference materials. You can gain access to documents (including Word, PowerPoint, Excel, PDF, etc.), audio clips, and even document sets.
Accessing the slides or accompanying paper from your favorite sessions has never been easier. Simply select the "AEA Library" link under the "Resources" menu and you can access all materials uploaded by your fellow AEA colleagues. If you would like to access documents specific to this year's conference, simply go to the "Evaluation 2009" link on the left. Here, you can rate materials to help other members, or even note a set of materials as a "favorite" for later ease of accessibility.
If you facilitated or presented at a workshop or session at Evaluation 2009
, please take advantage of this opportunity for greater exposure of your work and advancement of the field. As long as you hold copyright or have a copyright release, we welcome your materials. To submit your materials to the resource library, click on "Add Document(s)." Subsequently, for step 1, you will be required to submit information into specific fields, including title, description, and library category (e.g., Evaluation 2009
). It is important to click the save button after these fields have been filled. Afterwards, it's a matter of uploading your files, adding descriptive keywords, and selecting the appropriate TIGs.
So, go forth and have fun with the resource library. Because the system is still relatively new, please let me know if you come across any bugs or just have questions in general. I can be reached at [email protected]
|AEA's Financial Health |
|From Brian Yates, AEA Treasurer
Hi again! I'm your elected Treasurer, Brian Yates, and an AEA member since its inception. In my column in the April 2009 AEA Newsletter, I reported that our assets were $2.1 million. So, what are we doing with all that money?
First, we hold enough funds in FDIC-insured CDs so that AEA could function and recover fully if a major disaster struck. AEA experienced several severe financial challenges in its early history, including the 1989 Loma Prieta or "World Series" earthquake. The day before AEA was to hold its workshops and annual conference in San Francisco - at that time the only real "money maker" for the association - the earthquake hit nearby. Planes turned around, the airport closed, and those of us already in San Francisco dined by candlelight in a hotel without electricity! Ah, good times... Since then we have experienced other difficulties, as well, including a management firm that almost lost our financial assets and even our membership list.
Inspired by this history, AEA's financial policy has been to preserve sufficient assets to allow us to continue providing services to members even if a natural disaster or other emergency prevented us from holding our annual conference, pre- or post-workshops, summer institute, or all three. AEA fully insures itself to meet the largest budget from a year ago to a year ahead: currently $1.4 million. Having these assets in our restricted reserve provides better coverage than purchasing insurance. And it would have been critical if the H1N1 pandemic had prevented us from holding Evaluation 2009. This time, we were fortunate indeed!
Recently, I, other Board members, and many in AEA have noted that our mission should be not only to survive potential disasters, but also to use our resources to pursue goals consistent with our values as an organization. Absolutely! So, after assuring our immediate survival, we invest in activities such as training ourselves in new strategies for evaluation; fostering, maintaining, and expanding our own diversity; disseminating knowledge, facilitating formation of government policy formation, and nurturing new evaluators. Last fiscal year (July 08 - June 09) we spent $1.3 million doing just that, including over $0.1 million in new "pilot" projects such as our Evaluation Policy Task Force and our Volunteer Involvement Program. We also have initiated a full first-ever audit of AEA's financial operations, a voluntary effort as part of the board's new financial policy for our association. In these ways, AEA revenues are being spent to improve our profession and to help our services have greater impact in national and international arenas.
2009 Poster Exhibition Winners Announced
Congratulations to this year's winners of the AEA Poster Exhibition held in conjuction with Evaluation 2009
Poster #11, entitled Changing Horses Mid-Stream: Lessons Learned From Evaluator Transitions During Two Innovative Technology Experiences for Students and Teachers (ITEST) Science, Technology, Engineering, and Mathematics (STEM) Projects was presented by:
- Ginger Fitzhugh, Evaluation & Research Associates (shown, left)
- Karen Manuel, Puget Sound Center for Teaching, Learning and Technology
- Denice Hood, University of Illinois at Urbana-Champaign (shown, center)
- Mary White, Arizona Grants Management (shown, right)
- Mariann Farrell, Graphic Designer, Puget Sound Center for Teaching, Learning and Technology
"It is not uncommon for programs to change evaluators partway through a grant cycle by necessity or choice. Based on our experiences as incoming and outgoing evaluators of two multi-year, out-of-school time programs funded by the National Science Foundation, we considered the questions:
- What issues are associated with an evaluation transition for the evaluators, program staff, and other stakeholders?
- What strategies can facilitate a smooth transition for all stakeholders?
"We found the issues and strategies related to communication, context, logistics, and evaluation design. While many of these challenges and practices are common to all evaluations, some are especially important when there is a change in the evaluator."
The winning poster was judged for its visual appeal as well as written content. Speaking to the presenters afterward, they shared that they studied AEA's judging criteria closely and referred to it often. In honor of their first-place achievement, the team received free luncheon tickets to AEA's 2009 Awards Luncheon. They also will receive a $100 honorarium and a small gift from AEA.
|Art Supplies & AEA Donations Help Florida Teens Remain Drug-Free|
For the second year in a row, AEA teamed with a local non-profit during its annual conference to support a cause in that year's host city. This year, AEA partnered with Orlando's New Horizons Program - a comprehensive school-based substance abuse prevention/intervention program that for more than 30 years has actively worked with central Florida-area schools in an effort to provide counseling services and a life skills curriculum to benefit middle and high school students. The Center for Drug-Free Living's New Horizons Program has the distinct honor of earning official designation as a "model program" by the Florida Department of Children and Families, Substance Abuse Program Office.
New Horizons provides counseling services to teens referred by parents, teachers, guidance counselors, social workers, administrators or who seek help for themselves or their friends. The New Horizons program has been rigorously evaluated and evidence shows that students completing the program had improved school attendance and improved GPA, students reduced or eliminated their alcohol, marijuana, and other substance use. Students also decreased their risk factors and increased their knowledge of alcohol, tobacco, and other drugs.
The program's life skills curriculum covers drug abuse education, communication skills, decision making skills, coping skills, and anger management. Counselors provide substance abuse education and awareness activities through classroom presentations and school-wide projects. Counselors also promote suicide prevention awareness and violence free month. "Student substance abuse and personal problems greatly interfere with school performance as well as other areas of an adolescent's life," says Susan Nuckols, director of the New Horizons program. "It is important to make counseling services easily available to students in a school-based setting."
Prior to this year's Evaluation conference, attendees were notified of the community-based partnership with New Horizons and encouraged to bring office and craft supplies, as well as spare change and/or cash donations, to support the cause. The art supplies are useful in reinforcing positive messages. "We give them things to remind them to be drug-free and alcohol-free," explains Ingrid Todd, New Horizons Program Evaluator/Manager.
AEA donated a laser printer and 25 T-shirts and AEA attendees have donated more than $300 in cash contributions as well as assorted supplies ranging from reams of paper and pens to thumb drives and ribbons. Thank you! And, if you'd still like to contribute, you may do so by contacting [email protected]
or [email protected]
|Extension Education Evaluation TIG Presents Three Awards |
|The Extension Education Evaluation Topical Interest Group presented three awards at this year's Evaluation 2009 conference. They include:
Sustained Excellence in Extension Evaluation Award
Rama Radhakrishna, Pennsylvania State University
This award is given for conceptual, methodological and evaluation accomplishments making outstanding contributions to Extension and evaluation.
"Rama Radhakrishna is an outstanding evaluation expert and mentor to young professionals. He is an excellent professional who practices the program evaluation skills he teaches," said one letter of support. Another said, "His expertise in evaluation of Extension programs has been utilized to develop tracking systems and evaluation plans to show program outcomes." A third letter said, "Working with him is a pleasure. He is very receptive of ideas..." The Mary Nell Greenwood Extension Evaluation Award
Jewel Hairston, Virginia State University
This award is given for administrative or program leadership toward outstanding organizational, policy and resource contributions to Extension evaluation.
Jewel Hairston's letters of support specifically mentioned her administrative and program leadership in focusing on program evaluation as performance review criteria, supporting her faculty and staff in learning about evaluation with multiple resources and her evaluation capacity building planning for her colleagues at VSU and Virginia Tech. Excellence in Evaluation Training Award
Joseph L. Donaldson, University of Tennessee Extension
This award is given for evaluation training of outstanding quality by an individual or team. The nominees may be professional evaluator(s) or those whose major responsibility is not program evaluation. Criteria for excellence in evaluation training include creativity, enablement of trainees in evaluation practice, and the use of training to further extension evaluation.
Joseph Donaldson's nomination packet included information on the breadth and depth of his work in evaluation training, including his work on cost-benefit analysis and the planning, evaluation and reporting system. His letters of support described his integrity, professionalism, attention to detail and his obvious enthusiasm for the content that he helps others learn.
Congratulations to this year's recipients!
If your TIG has news to share, please contact AEA Communications Director Gwen Newman at [email protected]
|Scott Rosas Receives Presidential Citation for Outstanding Achievement|
|AEA member Scott Rosas, a senior consultant with Concept Systems, Inc., was one of five alumni recently honored by the University of Delaware (UD) during its annual Presidential Citation for Outstanding Achievement. Since its inception in 1992, the Presidential Citation program has honored UD graduates who have exhibited great promise in their professional and public service activities.
Rosas received a bachelor's degree in history, a master's degree in individual and family studies and, in 2002, a doctorate in human development and family studies. While at UD, he studied with Leslie Cooksy, the current president-elect of AEA and focused on conceptualization techniques for theory development and measurement in evaluation. He is a senior consultant with Concept Systems Inc., in Ithaca NY where he continues to apply the evaluation methodologies he explored as a graduate student at UD.
Rosas also served as an associate faculty member at the Bloomberg School of Public Health at Johns Hopkins University, and he joined the American Evaluation Association, where he serves as a leader within its Ethics Committee.
"The interdisciplinary program within the College of Education and Public Policy has allowed me to explore areas of interest to me and create a learning experience that has become the foundation for my work," Rosas said. "I truly value my time at the University -- its influence and impact on me has endured."
To share news of recognition, contact AEA Communications Director Gwen Newman at [email protected]
|Effective Grant Writing and Program Evaluation |
AEA members Francis K. O. Yuen, Kenneth L. Terao, and Anna Marie Schmidt are co-authors of a new book published by Wiley. Effective Grant Writing and Program Evaluation for Human Service Professionals
is described as a state-of-the-art guide for developing grants with a strong emphasis on using program outcome measurement to underscore need and accountability. From the Publisher's Website:
"Based on the authors' many years of experience in the public and nonprofit sectors, Effective Grant Writing and Program Evaluation for Human Service Professionals
integrates the topics of grant proposal writing and program evaluation, offering grant seekers the practical guidance they need to develop quality proposals, obtain funding, and demonstrate service results and accountability.
The authors clearly and succinctly illustrate and describe each stage of the grant writing and evaluation process. Problems or issues that arise frequently are highlighted and followed by specific advice. In addition, numerous real-world examples and exercises are included throughout the book to give readers the opportunity for reflection and practice."
From the Authors:
"Although there are many good grant writing or program evaluation books, our goal is to provide practitioners and students in human service a user-friendly, practical, and quality textbook that would allow them to acquire the essential and advanced knowledge and skills of grant writing and program evaluation. We wanted to target direct service practitioners who also have responsibilities as program planners, program managers, program evaluators, and agency administrators. However, its academic contents and easy-to-use plain language format allow this book to be used by graduate and undergraduate students in human services. It is both a practice text and an academic text."
About the Authors:
Francis K. O. Yuen is a Professor in the Division of Social Work at California State University, Sacramento. He has served in many capacities as human service agency administrator; principal investigator for government grant projects; grant writer and reviewer; and program evaluator for local, state, and international projects.
Kenneth L. Terao is a senior evaluation specialist with JBS International, Inc., Aguirre Division, a social science research firm located in the San Francisco Bay Area. He has been a director and evaluator for many major grant-funded projects in community and educational settings.
Anna Marie Schmidt is an evaluation specialist with JBS International, Inc., Aguirre Division, a social science research firm located in the San Francisco Bay Area. Previously, she managed national and international nonprofit projects. Go to the Publisher's Site
New Jobs & RFPs from the AEA Career Center
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:
- Request for Proposal (Evaluation Team) at International Labour Organization (Geneva, Switzerland)
- Research & Evaluation Associate at Edvantia Inc. (Nashville, TN, USA)
- Research Associate at American Institutes for Research (Washington, DC, USA)
- External Evaluation of the Regional Program Safe Cities (in Spanish) at UNIFEM (Brasilia, DF, Brazil)
- Lead Evaluator, Supervisory at Defense Language Institute Foreign Language Center (Monterey, CA, USA)
- Data Analyst at Excelsior College (Albany, NY, USA)
- Strategic Data Fellow at Center for Education Policy Research at Harvard University (Multiple Locations, USA)
- Research Associate II #9539 at University of Southern Maine (Portland, ME, USA)
- Associate Dean for Research at University of South Florica (Tampa, FL, USA)
- Senior Monitoring, Evaluation and Research Technical Advisor at IntraHealth International (Chapel Hill, NC, USA)
- Evaluation Research Advisor (50% time) at The Center for Victims of Torture (Minneapolis, MN, USA)
- Monitoring and Evaluation Manager, Uganda at ACDI/VOCA (Lira, UGANDA)
Descriptions for each of these positions, and many others, are available in the AEA Online Career Center. According to Google analytics, the Career Center received close to 4000 unique visitors in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee.
Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed.
Go to the AEA Online Career Center
Canadian Evaluation Society Call for Proposals
The Canadian Evaluation Society (CES) invites you to "Go Green, Gold and Global" at their annual conference in Victoria, British Columbia, on May 2 - 5, 2010.
The call for proposals for workshops and presentations is now available
along with more information about the event. The deadline for submissions is December 11, 2009 and registration for the conference will open early in 2010. The theme for this year's conference is Going Green, Gold, and Global: New Horizons for Evaluation
, which they elucidate as:
- Going Green - evaluation in the context of managing positive environmental change.
- Going Gold - evaluation of the impacts and effects of programs associated with the 2010 Winter Olympics and striving for excellence in evaluation.
- Going Global - evaluation of programs and policies in the context of British Columbia's international connections.
- New Horizons For Evaluation - exploration of emerging evaluation issues in both theory and practice, particularly in the three main themes
The 2010 CES conference organizers note: "CES encourages evaluators and related professionals to join this tradition of sharing a spirit of ever increasing openness to knowledge diversity. Evaluation professionals from governments, post-secondary institutions, private practice, non-profits, and the voluntary sector will come together to discuss, debate and learn from each other. CES members will discuss the latest developments in evaluation in Canada. As well, we invite non-members and evaluators from outside Canada to share information and their experience on evaluation initiatives." Go to the CES 2010 Conference Website
|The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275