|
|
| Newsletter: May 2010 | Vol 10, Issue 5
|
|
|
|
|
|
| Conference News & Bylaws Update |
Dear Colleagues,
 Greetings! I am just back from the Canadian Evaluation Society (CES) conference in beautiful Victoria, British Columbia. One of the big events of the conference was the rollout of the new CES credentialing program. Based on years of deliberation and development, the program is an exciting move in the professionalization of evaluation and one that I am sure will be a topic of conversation at our conference in November. Hearing about the new program was just one of many highlights of the CES conference. The integration of First Nations traditions in conference activities, a focus on the evaluation of environmental programs, and the student case competition are a sample of some of the others. Please help me to extend the same hospitality that I experienced to the CES incoming president, Martha McGuire, when she comes to San Antonio for the AEA conference.
Our conference theme of Evaluation Quality is generating discussion in a variety of venues. For some interesting (and humorous) commentary on the theme, go to John Gargani's blog. The Thought Leader discussions organized by Past President Debra Rog will also be turning toward issues of quality later in the year. And, stay tuned for announcements of our exceptional plenary speakers. I promise they will be both entertaining and thought-provoking (no, you old-timers, not Einstein!).
The Board continues to be busy. We are currently reviewing the comments that we received on the proposed revisions of the bylaws. A few members and one of the Priority Area Teams commented. Some expressed support, some identified concerns, and others asked for more information. All of the input was thoughtful and useful, and I appreciate the interest and time of those who participated. I anticipate that we will be able to complete our responses to your comments and have a final Board vote in time for this year's election. The revised bylaws will be included on this year's ballot along with the slate of board member and presidential candidates, coming out soon.
The Summer Evaluation Institute - one of the events that rival the conference in importance to the field - is also coming up next month. The Institute is a joint effort of AEA and the Centers for Disease Control and Prevention. It will be held in Atlanta, June 13-16. Go online for more information. We'd love to see you there!
Leslie Leslie Cooksy
|
|
 |
|
|
|
| Policy Watch - Promising Developments in Health Care Reform | |
From George Grob, Consultant to the Evaluation Policy Task Force
 Health care reform legislation is now yesterday's news, but the reforms themselves will be felt for years to come. A natural question for us is, "How will the reforms be evaluated?" We followed the development of the legislation quite closely and provided copies of the Evaluation Roadmap for a More Effective Government to key staff members of the congressional committees working on the legislation. We encouraged them to consider evaluation provisions in the bills leading up to the enactment of the Patient Protection and Affordable Care Act .
The health insurance reforms were originally lacking appropriate evaluation and oversight mechanisms. To be sure, there were (and still are) important requirements for evaluation ofadministrative simplification of health insurance transactions and for a survey of enrollee satisfaction with health plans offered through state health insurance exchanges. The Government Accountability Office is also required to review all aspects of the administration of the health insurance exchanges. However, overall, evaluation requirements of insurance reforms were sparse. We pointed this out to the relevant congressional staff. The final bill now includes oversight of the insurance reforms through the Office of the Inspector General of the Department of Health and Human Services. The Inspector General's oversight of all aspects of this title provides a framework for the conduct of independent evaluations as well as audits and investigations.
While the national debate raged over health insurance reform, less public attention was paid to the numerous public health and health systems reforms that were contained in the bill. Only one sixth of the law, Title I, Quality, Affordable Health Care for All Americans, deals directly with health insurance reforms. Title II, Role of Public Programs, adds health insurance improvements through public programs, but also includes demonstrations of innovative health care delivery and financing mechanisms, primarily Medicaid. Fully two-thirds of the remaining pages of this 2400 page bill, titles III through X, address public health programs and health care systems reforms.
Fortunately, these public health and health systems reforms were drafted with evaluation in mind. Starting with title II, requirements for the kinds of evaluation that the Roadmap calls for are prolific, including evaluations of new demonstrations authorized under Medicaid, health care indicators, quality improvements, community health, nursing home reforms, improvements in coverage of prescriptions drugs, health care workforce enhancements, prevention of chronic diseases, improved access to medical therapies, and preventive health. To get a feel for this, I suggest you pull up the pdf version of the bill at the link provided above and do a search with the partial word "evaluat."
We cannot take credit for this. The culture of evaluation within the public health community is largely responsible for the successful embedding of evaluation throughout the bill. Indeed, it is encouraging to see this development, a reflection that evaluation may be coming of age.
I will share more about this in future editions of the AEA newsletter and on the evaluation policy discussion list.
Go to the Evaluation Policy Signup page to join the discussion list |
| Meet Rakesh Mohan - AEA Board Member & Government Employee |
AEA's 5,700 members worldwide represent a range of backgrounds, specialties and interest areas. Join us as we profile a different member each month via our Questions and Answer column. This month's profile spotlights Rakesh Mohan, an active long-time member who currently sits on the AEA board and interfaces with the state legislature of Idaho.
 Name: Rakesh Mohan
Affiliation: Director, Office of Performance Evaluations, Idaho State Legislature
Degrees: Master of Public Administration (MPA)
Years in the Evaluation Field: 21 years
Joined AEA: 1992
AEA Leadership Includes: Board Member (2008-2010); Editorial Advisory Board, New Directions for Evaluation; Former Chair, Publications Committee and State and Local Government Evaluation TIG
Why do you belong to AEA?
"AEA is a professionally and demographically rich and diverse organization that welcomes anyone interested in evaluation. It offers a broad spectrum of opportunities for learning, teaching, and networking. I attended my first AEA conference in Seattle in 1992, having only a little familiarity with program evaluation. At the conference, I was filled with awe - like trying to drink water from a fire hose. Initially I knew no one in AEA, but I started coming into contact with some wonderful people who were always willing to listen and give me opportunities to learn and grow as a professional. Many of them have become good friends. Over the years, I have had the honor and privilege of serving AEA in various capacities starting with program chair for the State and Local Government TIG in 1994. I have participated in every AEA conference since 1994."
Why do you choose to work in the field of evaluation?
"I accidentally entered the field of evaluation in 1988 when I received the only job offer with the Kansas legislative performance audit shop in my effort to move away from working in a chemistry lab. At that time, I had never heard of that office, nor did I know anything about performance auditing or program evaluation. Gradually I began to understand and like the field of performance auditing, which for the most part is similar to program evaluation. Being an evaluator can be more than a job. One can make it a lifelong vocation of working toward the betterment of society. As an evaluator, on every project, you are given an opportunity to make a difference, to make an impact."
What's the most memorable or meaningful evaluation that you have been a part of - and why?
"I am still searching for that MOST memorable or meaningful evaluation. For the past seven years, I have been involved with many challenging and politically sensitive evaluations in public education, health, transportation, and corrections. Every year, I am dealing with increasingly more complex issues that are also more politically sensitive - some of them are littered with cow-pies or, even worse, with landmines; you need to make sure where you step. Though highly stressful, such projects also offer a great deal of professional challenge and satisfaction."
What advice would you give to those new to the field?
"Advice? I have still much to learn, and besides that, I am too young to give advice. But if I must, my advice would be: Don't be intimidated by big names in AEA; they are some of the kindest and nicest people you will ever meet. Reach out to them and you will be glad you did."
If you know someone who represents The Face of AEA, send recommendations to AEA's Communications Director Gwen Newman at gwen@eval.org. |
| TechTalk - Disseminate Knowledge - By Commenting | | From LaMarcus Bolton, AEA Technology Director
Back in January, I had the pleasure to chat with John LaVelle, the coordinator for our new AEA365 blog. These blog posts have helped many members connect, learn, and improve their practices. Although the daily posts are the central component to AEA365, an often overlooked, but essential, element of a successful blog is the interaction and participation of those who also take time to comment.
Commenters help to ensure a more lively discussion by adding context, supporting details, and sometimes contributing new (or challenging) perspectives. Because AEA members not only possess a wealth of knowledge but are continuous learners as well, I wanted to discuss some of the ways that you can get started with and become more involved in the dissemination and acquisition of knowledge via active commenting.
1. Blog commenting etiquette: If you have limited experience participating in blogs, take some time out initially to review some tips for "ideal" blog commenting. For instance, some of the examples noted from the Lost Art of Blogging states that commenters should seek to be consistent, succinct, polite, and contribute value at the same time.
2. AEA365 blog commenting: After reviewing the full list of blog commenting etiquette tips, jump right into commenting on your favorite posts. To comment, simply click the "Comment" link directly under the post's title, scroll down to the "Leave a comment!" section, fill in your appropriate information, and comment away. If you really want to take commenting to the next level and be a little more personal and memorable, adding a free Gravatar will allow you to automatically insert your photo to all of your comments.
3. Subscribe: It is quite challenging to be a consistent commenter and stay engaged in comment-based discussions if you have no way of knowing when new comments are added. So, we have two subscription options for comments (as well as for the blog itself)-either subscribing via RSS or via email. For more information, see the "Take Action" pane on the right side of the AEA365 blog.
4. Share with others: Lastly, if you come across one intriguing post that you think a colleague would enjoy, use the "ShareThis" button beneath the blog post to circulate the post via the most popular mediums (e.g., email, Facebook, Twitter, LinkedIn, etc.). Doing so results in more exposure and ultimately brings in unique and valuable contributions.
I hope these tips will help you all take a more active role in AEA365's ongoing comment-based discussions. AEA has enhancements planned for the future that should further enrich your commenting experience, so stay tuned! And, in the meantime, if you run into any issues, or have general questions, comments, or suggestions, please do not hesitate to contact me at marcus@eval.org. |
| Interview with Ray Rist, Advisor to the World Bank | |
 Ray C. Rist is currently an advisor on evaluation capacity building to the World Bank and to governments across the globe. He has authored or edited 27 books and lectured in 82 countries. He has held senior positions in both the Executive and Legislative branches of the U.S. government. He is currently President of the International Development Evaluation Association (IDEAS) and Co-Director of the International Program For Development Evaluation Training (IPDET). Rist recently agreed to an interview as part of AEA's series of profiles of leaders in international evaluation, with excerpts below.
Rist's introduction to evaluation he describes as a slow and steady work in progress.
"I gradually moved from a focus on urban sociology and urban anthropology into the study ... of programs attempting to address these problems... My role has evolved over the years -- from university-based evaluator/researcher, to evaluation manager in the U.S. federal government, to evaluation manager in the World Bank, to someone emphasizing capacity building for both individuals and organizations in developing countries."
Rist is excited about an increased focus on performance-based or results-based management and measurement in developing countries - and increased capacity within.
"There is much more skill and expertise in developing countries than a decade ago, to the point now where in developing countries, there are evaluations undertaken that have no developed country input or involvement."
Rist would like to see more of an international focus on monitoring - and sharing of experiences.
"Evaluators tend to avoid monitoring as both a public sector function and as a source of important data for their own craft...And, there is, as it were, the decomposition of evaluation into such disparate and distinct sub-specialties -- the folks doing evaluation work in the HIV/AIDS arena and those working in climate change, for example, have little or nothing to say to one another, and yet they are both global challenges where national responses are clearly inadequate."
Building capacity and expanding networks are priority areas, both for IDEAS and IPDET.
"The list-serve for IPDET, for example, has more than 3,200 members in at least 125 countries who are in contact constantly and putting their experiences and questions up for comment and reaction from others in the IPDET community of graduates. IDEAS now has more than 1050 members in 105 countries. These are global communities that are both addressing the micro and macro issues of evaluation. This is absolutely an exciting time to be engaged in development evaluation."
Special thanks to members of AEA's International Committee for its help in producing this series of international profiles.
Go to the full transcript of the interview with Ray Rist |
| Summer Evaluation Institute June 13-16 in Atlanta, Georgia |
Join the American Evaluation Association and the Centers for Disease Control and Prevention (CDC) as they celebrate the 10th anniversary of Summer Institute June 13-16 in Atlanta with three keynote speakers each nationally acclaimed and equipped to address the age-old challenges of health care evaluation, a new era of performance evaluation and the allure - and effectiveness - of social networking for social change.
Janet Collins on Monday will talk about the challenges of program evaluation in public health and at CDC, some of the opportunities for increasing the leverage and impact of program evaluation and performance measurement, and some key current initiatives and new directions being pursued at CDC as examples of how one large organization can effectively employ performance measurement and program evaluation for program improvement. Collins joined CDC in 1990 and, for the past four years, has served as Director of CDC's National Center for Chronic Disease Prevention and Health Promotion, directing a diverse portfolio of programmatic and scientific initiatives across ten Divisions. Collins is a respected voice in program evaluation with the public health community; she earned her Ph.D. in Educational Psychology from Stanford University.
Media maven Allison H. Fine on Tuesday will discuss the challenges and opportunities for measuring social media efforts in a networked world. Founder of Innovation Network, a nonprofit dedicated to transforming evaluation for social change, Fine is also author of the award-winning book Momentum: Igniting Social Change in the Connected Age and a senior fellow at Demos: A Network for Change and Action in New York City. Allison hosts a monthly podcast for The Chronicle of Philanthropy called "Social Good" and writes her own blog, A. Fine Blog. Her newest book, The Networked Nonprofit, with Beth Kanter, is due out in June 2010 from John Wiley and Sons.
George Washington University's Kathryn Newcomer on Wednesday will describe the logistical challenges and skepticism that the Obama Administration inherited, lay out initial actions taken, new commitments articulated, and unique challenges faced as new agency leaders move performance improvement efforts forward. Newcomer is Director of the Trachtenberg School of Public Policy and Public Administration and Co-Director of the Midge Smith Center for Evaluation Effectiveness, home of The Evaluators' Institute (TEI). She has been named an outstanding teacher and has published numerous journal articles as well as five books, including Meeting the Challenges of Performance-Oriented Government, Getting Results: A Guide for Federal Leaders and Managers, and Transformational Leadership: Leading Change in Public and Nonprofit Agencies.
The AEA/CDC Summer Evaluation Institute is aimed at professionals who conduct or manage evaluations as well as those who use results for program improvement. Evaluators from any level of government, staff of nonprofit and community organizations, applied researchers, grant makers, foundation program officers, and social science students are encouraged to attend. Presenting are experts who have conducted evaluations in a variety of settings, nationally known authors, those working on the cutting edge, evaluation experts and outstanding trainers.
Each keynote address is scheduled to begin at 8:30 a.m.
Go to the Summer Institute page to learn more or to register |
| Evaluation Across the Disciplines - David Fetterman |
AEA members practice in an amazing array of contexts. We're hoping to highlight how your work is making a difference in a rich variety of places and spaces.
 AEA member David Fetterman published an article this month titled Empowerment Evaluation: A Collaborative Approach to Evaluating and Transforming a Medical School Curriculum. The study findings were published in Academic Medicine (Volume 85(5)), a refereed journal of the Association of American Medical Colleges. The study describes how empowerment evaluation was applied in the Stanford University Medical School, including such tools as:developing a culture of evidence, using a critical friend, encouraging a cycle of reflection and action, cultivating a community of learners, and developing reflective practitioners. It is an approach that places the evaluation in the control of the community members and is designed to help people help themselves.
The study concluded that applying empowerment evaluation to curriculum development contributed to improvements in course and clerkship ratings. "In comparing evaluation results before and after stakeholders began using this approach, we found that the average student ratings for the required courses (18 courses) improved significantly (P=.04; Student's one-sample t test).Similarly, ratings for the most of the required clerkships remained steady or improved" (p. 817). David was AEA's featured Thought Leader for the past week on the Thought Leaders Discussion Forum. To view the Thought Leader's archives, and/or to participate in future discussions, sign in to the AEA website and choose "Thought Leaders Discussions" from the "Members Only" menu at the top.
Sign on at: http://eval.org/ Your Username: Your Password:
If you have a recently published article or other experience to share highlighting how evaluation is making a difference across the disciplines, contact Gwen Newman, Communications Director, for possible inclusion in a future newsletter gwen@eval.org. |
| Advanced & Multivariate Statistical Methods for Social Science Research |
 AEA member Soleman H. Abu-Bader is the author of Advanced & Multivariate Statistical Methods for Social Science Research, a new book published by Lyceum Books.
From the Publisher:
"Unlike other advanced statistical texts, Advanced & Multivariate Statistical Methods for Social Science Research combines the theory and practice behind a number of advanced and multivariate statistical techniques which students of the social sciences need to evaluate, analyze, and test their research hypotheses. Each chapter discusses the purpose, rationale, and assumptions for using each statistical test, rather than focusing on the memorization of formulas. The tests are further elucidated throughout the text by real examples of analysis. Of particular value to students is the book's detailed discussion of how to utilize SPSS to run each test, read its output, interpret, and write the results."
From the Author:
"Advanced & Multivariate Statistical Methods for Social Science Research is an indispensable resource for students of disciplines as varied as social work, nursing, public health, psychology, and education. The book reviews basic bivariate tests (correlation, student's t-tests, analysis of variances, and chi-square) and discusses methods for data evaluation including missing cases, univariate and multivariate outlier cases, and normality of distributions. The book then discusses regression analysis (simple, multiple, and logistic), factorial analysis of variance and covariance, repeated measures analysis of variance (within-subjects ANOVA and between-within-subjects ANOVA), multivariate analysis of variance and covariance, and canonical regression analysis (multivariate multiple regression)."
About the Author:
Soleman Abu-Bader received his B.A. in social work from Ben-Gurion University in 1992, and his M.S.W. from Augsburg College in Minneapolis, MN in 1994. In 1998, Abu-Bader completed his Ph.D. at the University of Utah College of Social Work, Salt Lake City, UT. He joined the Howard University School of Social Work Faculty in Fall 2000. He has written and presented research papers on work satisfaction and burnout among social workers, depression, life satisfaction, and self-esteem of frail elderly, former welfare recipients, and among Arab Americans. He also is author of Using Statistical Methods for Social Work Practice and serves on the Editorial Board of the Journal of Immigrant and Refugee Studies.
Go to the Publisher's Site |
| Many Ways to Contribute to AEA - Cultivating Our Cultural Capacity | |
Looking for ways to get involved in the life of the association? AEA's new Member Involvement Initiative (MII) has the following updates related to new volunteer opportunities within AEA:
Cultivating Our Cultural Capacity Working Group: The CCC working group will help to identify, prioritize, and plan for programs to assist graduate students and evaluators within three years of the start of their professional practice who share an interest in culture and representation within the field of evaluation. Chaired by Multiethnic Issues TIG Co-chair Tamara Bertrand Jones, the group will brainstorm, prioritize, and plan for ways to foster the leadership pipeline, build professional networks, and meet the professional development and career progression needs of evaluators of color and evaluators who wish to work cross-culturally. If you are a graduate student or a professional within your first three years of professional practice and are interested in participating, please send an email by Friday, June 18, to Susan Kistler at susan@eval.org noting your particular interests in this area. The group will convene for an estimated one year, meeting approximately once per month via conference call and conducting our discussions primarily via email exchange.
|
|
New Jobs & RFPs from the AEA Career Center | |
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:
- Associate Director, Division of Evaluation and Research at Louisiana Public Health Institute (New Orleans, LA, USA)
- Research Intervention Coordinator at Flexible Work & Well-Being Center, Minnesota Population Center (Denver, CO, USA)
- Senior Monitoring and Evaluation Specialist at Management Systems International (Washington, DC, USA)
- Social Science Research Analyst at USDA - Food and Nutrition Service (Alexandria, VA, USA)
- Program Evaluator at Kauffman & Associates Inc. (Silver Spring, MD, USA)
- RFP - Evaluation of Performance Measure Use at National Quality Forum (Washington, DC, USA)
- Evaluation Grant Opportunity at Ruddie Memorial Youth Foundation (USA)
- Regional Evaluator (Senior Research Associate) at Education Development Center (Newton, MA, USA)
- Senior Advisor, MEASURE Project at PATH (Calverton, MD, USA)
- Summative Evaluation RFP at Northeast Regional Resource Center (Williston, VT, USA)
Descriptions for each of these positions, and many others, are available in the AEA Online Career Center. According to Google analytics, the Career Center received over 4000 unique visitors in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee.
Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed.
Go to the AEA Online Career Center |
| Get Involved | | Get the most from your membership by taking advantage of the many things that you can do right now to participate in the life of the association, share your input, and promote your business.
|
| About Us |
| The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
The American Evaluation Association's mission is to:
- Improve evaluation practices and methods
- Increase evaluation use
- Promote evaluation as a profession and
- Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275
| |
|
|
|
|
|
|
|
|
