Newsletter: February 2010 Vol 10, Issue 2

Banner600

Your Input Wanted on Three Key Documents
Dear AEA Colleagues,
 
CooksyGreetings from snowy Delaware! Because of the recent blizzard, the Board's planned meeting in San Antonio was conducted via phone and webinar. Despite the less than ideal circumstances, we had a productive meeting. Among other items, the Board discussed three significant documents, each of which will be circulated for member input soon.
 
The first document is a statement on Culturally Competent Evaluation, prepared by the Task Force convened by the Diversity Committee (now a part of the Values Priority Area Team). The second document, the Evaluation Road Map for a More Effective Government, was written by the Evaluation Policy Task Force for use in meetings with federal policy makers. Thirdly, the Board has been revising the Association By-laws for the past few months in order to update them and bring them into congruence with the policy-based governance model. After a feedback period and revisions in response to input, the By-laws will be submitted for a vote by the membership in June. The other two documents will also be revised based on member comments and then resubmitted to the Board for final approval as Association statements.
 
Since the last newsletter, the Call for Proposals for Evaluation 2010 has been released. While I hope that the conference theme of Evaluation Quality will inspire many proposals on the various ways we think about the quality of our work, remember that your submissions do not have to directly relate to the theme.
 
The Call for Nominations for Board Members At Large, the Treasurer, and President-Elect has also been issued. Please consider nominating someone (self-nominations are accepted). Serving on the Board is a great experience. For me, the opportunity to work with a diverse, talented, and dedicated group of AEA members on the Board is one of the greatest rewards of being President. (For a list of current Board members, click here.
 
There are a lot of ways to participate in AEA these days. Because of the particular importance of the Culturally Competent Evaluation statement, the Evaluation Roadmap, and the revised by-laws in AEA's evolution, I hope you will take advantage of the opportunities for member input. I am also looking forward to the variety of proposals for conference sessions; and I encourage you to nominate yourself or someone else for the AEA Board.
 
Thank you for all the ways you contribute to our community!
Leslie
Leslie Cooksy
2010 AEA President
In This Issue
Policy Watch with George Grob
Donna Mertens Honored
Evaluation Ranked a Top Profession
Thought Leader Rodney Hopson
Meet Nick York, UK Department for International Development
TechTalk with LaMarcus Bolton
New Journal: Critical Education
Book: Oral History for the Qualitative Researcher
EERS Conference April 18-20
New Volunteer Ops
New Jobs Postings
Get Involved
About Us
Quick Links
Policy Watch - Evaluation Policy in the President's Budget
From George Grob, Consultant to the Evaluation Policy Task Force
 
Grob

When President Obama took office a little more than a year ago, we noticed his interest in program evaluation. Now we have fresh evidence of it in his 2011 budget. AEA's Executive Director, Susan Kistler, has prepared an analysis of that budget, which we are sharing with you here. Evaluation is woven throughout the document. Here are the highlights.

 

Evaluation of specific Federal programs. This includes proposals to increase the number of effective teachers and principals; boost development of clean energy on tribal lands; improve services for seniors and people with disabilities; and promote wellness initiatives in the Federal workplace. Especially noteworthy is a request for $500 million to expand the Investing in Innovation Fund, to evaluate and expand proven models for achieving student success.

 

Terminations, reductions, and savings. The budget cites evaluations to support budget reductions in several programs, such as certain children and families services' job demonstration programs, rural health care services programs, and watershed and flood prevention programs.

 

Sweeping evaluation policy. Far and away, the most impressive evaluation message is a set of overarching policies found in an annex to the budget called Analytical Perspectives. The parts that are relevant to evaluation are the first two sections (labeled 7 and 8) in a chapter called "Performance and Management."

 

Section 7, Delivering High-Performance Government, provides a new look at performance measurement and substitutes a new performance management system that builds on but also replaces the PART system used by the previous Administration.

 

Section 8, Program Evaluation, provides detailed steps and explanations to implement the policies enunciated by OMB Director Peter Orzag in his October 7 memorandum and discussed in our October column. The introductory paragraph puts evaluation in the context of performance management. "Performance measurement is a critical tool managers use to improve performance, but often cannot conclusively answer questions about how outcomes would differ in the absence of a program or if a program had been administered in a different way. That is where pro­gram evaluations play a critical role."

 

The second paragraph is equally telling. "A central pillar of good government is a culture where answering . . . questions [about how well programs work] is a fundamental part of program design and where agencies have the capacity to use evidence to invest more in what works and less in what does not."

 

The budget allocates approximately $100 million to 17 agencies to conduct new evaluations with strong study designs that address important, actionable questions or to strengthen agency capacity to support such strong evaluations.

 

We cannot do justice to these new policies in this short article. I urge evaluators to read these two sections themselves. For the evaluators involved in the work of the 17 agencies named in section 8, my take on this is that the ball is now in your court. Actually, I think it is in all of our courts.

 

Go to the EPTF web page

Donna Mertens Honored with AEA's 2009 Lazarsfeld Theory Award

Mertens4The American Evaluation Association honored Donna M. Mertens with its 2009 Paul F. Lazarsfeld Theory Award. The work and influence of Mertens, a Washington, DC-based educator and author, have been instrumental to the field as well as the community.

 

Mertens posits that the field can transform society through evaluation work that shares - or brings - the voices of those pushed to societal margins into the world of research. This applies to those discriminated and oppressed due to factors including but not limited to race/ethnicity, disability, immigrant status, political conflicts, sexual orientation, poverty, gender or age as well as power structures that perpetuate social inequities and indigenous people and scholars from marginalized communities undergoing change.

 

"Their voices, shared with scholars who work as their partners to support the increase of social justice and human rights, are reflected in this shift to transformative beliefs to guide researchers and evaluators. The transformative approach to evaluation makes explicit the use of evaluation for the purpose of furthering social justice," says Mertens. "This focus in our theory and practice is consistent with the ethical codes that guide our profession. I am thrilled to be honored in this way and humbled to think of my small part in contributing to this critical aspect of theory in evaluation."

 

A prolific writer and author, Mertens served as AEA president in 1998 and helped found AEA's Diversity Initiative, its Graduate Education Diversity Internship Program (GEDIP) as well as the International Organization for Cooperation in Evaluation (IOCE). She has made lifelong contributions to evaluation theory through teaching, presenting and practice and has authored/edited several books, most recently Research and Evaluation in Education & Psychology: Integrating Diversity with Quantitative, Qualitative, and Mixed Methods (SAGE, 2010), Transformative Research and Evaluation (Guilford Press, 2009) and Handbook of Social Research Ethics (with Pauline Ginsberg, co-editor, SAGE, 2009).

 
Mertens is a professor in the Department of Educational Foundations and Research at Gallaudet University and teaches research methods and program evaluation to graduate-level deaf and hearing students in multiple programs, including education, administration, psychology, social work, audiology, and international development. The major focus of her work is the blending of issues of social justice and human rights with research and evaluation frameworks and methods.  
 
Kiplinger Ranks Program Evaluators as a Top Career for the Next Decade

Earlier this month, Kiplinger ranked program evaluation as one of the top jobs for the next decade - one offering financial sustainability as well as social acceptability. We took a moment to touch base with those in the field to get their reaction to what Kiplinger called a baker's dozen of professions that promise income growth, work-life balance and social impact as well as some personal insight into their day-to-day work.

"Being an evaluation consultant is a trusted position and one that holds a great deal of influence," says Jara Dean-Coffey, founder and principal of jdcPartnership in San Rafael, California. "It is a privilege and honor to work with those who have committed themselves to making the world a better place. And, it is inspiring to see the value of rigorous inquiry and assessment rising to the top of the list as important skills," she notes. "The world has been and continues to change rapidly with increasing diversity and complexity. The ability to step back so that one can determine if and how to step forward, sideward or in some cases, stand still seems incredibly valuable at this stage of development as a global community."

David Bernstein, a senior study director at Westat and a career professional with almost three decades of experience, shares his reaction with a sense of humor - and pride - as he addresses two key points.

"I would recommend program evaluation as a career for those with an insatiable curiosity, a high tolerance for occasional tedium, an ability to be flexible yet willing to adhere to high professional standards, and a desire to do socially redeeming work that can truly make a difference.

"Status? I suppose our work has a certain status with policy wonks. Program evaluators in general are probably better educated than those in many other careers, and I suppose that carries a certain amount of status. However, I have yet to hear my children or their friends say, "Wow! Your dad is a program evaluator? That's cool!!!

"Socially redeeming? This is one of the more redeeming attributes of a career as a program evaluator. Every time I look at AEA's Guiding Principles, I am filled with pride, and provided with a healthy reminder that our work makes a difference."

To read the original article citing program evaluation as one of the most enduring professions for the next decade, click here. And best of luck with your own personal and professional journey!

Rodney Hopson to Headline Thought Leaders Series in March

HopsonWhat a great kickoff for the 2010 Thought Leaders Discussion Series. We heard from Jane Davidson in January and Ricardo Millet in February. And for March? Rodney Hopson will be joining us in the Thought Leader's Forum the week of March 7-13, 2010.

 

Rodney Hopson has served on the AEA Board of Directors and was the founding chair of the Graduate Education Diversity Internship Program. He has helped the association and the field to explore issues related to i) research on evaluation, especially in a recent National Science Foundation study on logic model use (with Rosalie Torres), ii) study and application of program evaluation standards for the third edition of Program Evaluation Standards (with Don Yarbrough, Lyn Shula, and Flora Caruthers), and iii) understanding the role of culture and context within evaluation and related disciplines. Hopson is the Hillman Distinguished Professor in Sociolinguistics, Ethnography, and Evaluation, Department of Educational Foundations and Leadership in the School of Education, and faculty member in the Center for Interpretive and Qualitative Research, Duquesne University.
 
For those new to this program, the Thought Leader Discussion Series brings together members for an informal week-long asynchronous discussion around issues raised by a guest thought leader. The full exchange takes place electronically via the AEA Thought Leaders Forum (formerly the President's Forum) and postings may be checked at your convenience each day throughout the week.
 
How does one sign up for the AEA Discussion Series? This is a members-only opportunity and it is free to all members who wish to participate. To join the forum, sign on to the AEA website using your username and password, navigate over to the forums, and click on "Add/Change Subscriptions" near the top right of the forum page. 

Login at: http://www.eval.org

Your username:
Your password:
 
If you have questions about the forums, or run into technical difficulties, please check the Thought Leaders Frequently Asked Questions and then do not hesitate to contact us here in the AEA office at [email protected].
Interview with Nick York, UK Department for International Development

York3Nick York, head of evaluation for the United Kingdom Department for International Development (DFID) and chair of the Organization for Economic Co-operation and Development's Development Assistance Committee's evaluation network, studied economics at Cambridge and joined the UK government economic service in 1994. He spent 10 years on health systems reform in the UK, helping to develop the international evidence base for quality in health care. He joined DFID in 2004 where he has played a leading role in developing the UK government's evaluation policy stance and has helped to deliver major evaluations in areas such as budget support, gender, country programs, HIV and health. York has a special interest in developing international systems for rigorous impact evaluation and helped lead the Network of Networks on Impact Evaluation (NONIE) as well as 3ie, a new organization based in Delhi that funds new impact evaluations to help promote development effectiveness. He recently agreed to an interview as part of AEA's series of profiles of leaders in international evaluation.

York shares that he got into evaluation "quite by accident," that he currently is engaged in capacity building, and is intrigued with the real-world impact of evaluation.

"I moved into it from health systems reform and evidence-based health care. The UK Department for International Development (DFID) urgently needed to build up their evaluation capacity and hired me.

"Helping some of the poorest people in the world is pretty motivating, but the bit I would highlight is evaluation partnerships, across many countries and organizations. I am involved in the Organization for Economic Co-operation and Development's Development Assistance Committee's evaluation network (OECD DAC) and it is very satisfying to see things moving forward through a collective effort from many people."

York's work entails complex evaluations with multiple partners. While immensely gratifying, collaboration can also be extremely challenging - and insightful.

"Getting people from different backgrounds and disciplines to value each other's contributions can be challenging. People who work in evaluation, particularly in international development, are really passionate about their work and develop their expertise over many years.

Next steps, he says, are broadening the circle of engagement and the sphere of influence.

"I am most interested in how we can learn from others. For example, what do the experiences from evidence-based health care and medicine have to tell us in international development? How have others managed to measure policy dialogue and influencing? How can we promote the impact of evaluation evidence on decision making?"

This is one in a series of interviews AEA is conducting with leading figures in the international evaluation community. Many thanks to AEA's International Committee for its help with these interviews.

Go to the full interview

TechTalk - AEA Website Enhancements and Improved Navigation 
From LaMarcus Bolton, AEA Technology Director
 
Bolton

Over the past several months, AEA has made great strides in keeping up with new technological developments and enhancing our online community and overall member connectivity. To name a few, this includes our aea365 tip-a-day blog (discussed last month), webinar demonstration series, thought leader discussion series, evaluation headlines, and our always-expanding online message forums.

 

To improve accessibility to our expanded content offerings, I am happy to say that AEA has recently upgraded the navigation of our website to make site-browsing more user-friendly. Particularly, the new layout's headings are now topically-based, which makes finding information much more intuitive. For example, we've changed the "Publications" heading to "Reading" and include more of our community-driven content (e.g., public eLibrary, aea365, etc.). Secondly, we've implemented a similar modification, changing the "Training" heading to "Learning", which now encompasses not only our physical learning venues, but our virtual ones as well (e.g., webinar demonstration series, thought leader discussion series, etc.). Thirdly, all of our general online community information (e.g., EVALTALK discussion list, groups and forums subscriptions, LinkedIn group, etc.) can be found within the "Community" header. This makes finding and connecting to our networking communities much easier. Fourth, in case you're starting your career as an evaluator, or wishing to find one, our "Career" and "Find an Evaluator" tabs are more straight-forward for your respective needs. Lastly, our new "Members Only" heading is now essentially a one-stop-shop for all things membership-related. This includes not only membership management information, but also gives direct access to our exclusive members-only benefits.

 

While these may be minor changes, I'm sure they will be significant in enhancing your overall browsing experience. Please stay tuned as we progressively improve our website and membership offerings. In the meantime, if you have any questions, suggestions, or concerns, please do not hesitate to contact me at [email protected].

New Journal: Critical Education

Senior AEA member Sandra Mathison and her colleague E. Wayne Ross are founding co-editors of an online journal, Critical Education. AEA members Melissa Freeman, Saville Kushner and Linda Mabry are part of the journal's editorial team. Critical Education is an international peer-reviewed journal focusing on critical examinations of contemporary education contexts and practices. Critical Education is interested in theoretical and empirical research as well as articles that advance educational practices that challenge the existing state of affairs in society, schools, and informal education.

 

AEA members may be particularly interested in the journal's very first article, as it focuses on High Stakes Testing, an issue tackled by AEA via the association's public statements on Educational Accountability (2006) and High Stakes Testing (2002). The premier article in Critical Education is "The Idiocy of Policy: The Anti-Democratic Curriculum of High-stakes Testing" by Wayne Au. Au is assistant professor of education at Cal State University, Fullerton and author of Unequal By Design: High-Stakes Testing and the Standardization of Inequality (Routledge, 2009).

 

Notes Mathison, "We saw a need for an open access journal that encourages a wide range of critical research perspectives in education and while this isn't an evaluation journal, I believe many of the topics explored will be of interest to evaluators working in educational contexts."

 

To receive notification of new content in Critical Education, sign up as a journal user on their website.

 

Go to the Critical Education website

Oral History for the Qualitative Researcher: Choreographing the Story 

JanesickAEA member Valerie J. Janesick is author of a new book Oral History for the Qualitative Researcher: Choreographing the Story, published by Guilford Press.

From the Publisher's Site:

"Oral history is a particularly useful way to capture ordinary people's lived experiences. This innovative book introduces the full array of oral history research methods and invites students and qualitative researchers to try them out in their own work. Using choreography as an organizing metaphor, the author presents creative strategies for collecting, representing, analyzing, and interpreting oral history data. Instructive exercises and activities help readers develop specific skills, such as nonparticipant observation, interviewing, and writing, with a special section on creating found data poems from interview transcripts. Also covered are uses of journals, court transcripts, and other documents; Internet resources, such as social networking sites; and photography and video. Emphasizing a social justice perspective, the book includes excerpts of oral histories from 9/11 and hurricane Katrina, among other detailed case examples."

 

About the Book:

"I have always been interested in history and as a qualitative researcher, I was absorbed in internet inquiry and all the ethical implications of doing research on the web, specifically through blogs, or social networks. What sets this apart from other texts," says Janesick, "is my focus on internet inquiry and its implications, and the use of poetry to represent interview data in the form of found data poems. Found data poems are constructed from the words in the data. In addition, I see oral history as part of a larger social justice project to document the lived experience of those members of society who do not often get an opportunity to tell their stories. I hope this book will advance oral history in terms of going beyond the standard interview." 

 

About the Author:

Valerie J. Janesick is a Professor of Educational Leadership and Policy Studies at the University of South Florida, Tampa. She teaches classes in Qualitative Research Methods, Curriculum Theory and Inquiry, Foundations of Curriculum, Issues in Curriculum, and Ethics and Educational Leadership. Her text, Stretching Exercises for Qualitative Researchers, 2nd edition, (2004) SAGE Publications, includes ways to integrate the arts in qualitative research projects. Her writings have been published in Curriculum Inquiry, Qualitative Inquiry, Anthropology and Education Quarterly and other major journals. Janesick serves on the Editorial Board of Educational Researcher, The Journal of Curriculum and Pedagogy and The Qualitative Report. Her most prized possession is her British Library Reader's Card as she is working on an archival project on the letters of John Dewey written to educators around the world. Find her blog on Critical Pedagogy and poetry at: http://freire.mcgill.ca/content/critical-pedagogy-and-poetry.

 

AEA members receive 20% off the retail price of all books and journals ordered directly from Guilford Press as part of AEA's Publishing Partners program. To receive your 20% discount, use the promotional code "AEA" online or call 1-800-365-7006.

 

Go to the Publisher's Site

EERS Conference Scheduled April 18-20 in New Jersey

The Eastern Evaluation Research Society (EERS) will hold its 33rd annual conference April 18-20, at the Seaview Resort and Spa in Galloway, NJ (just north of Atlantic City). This year's theme is Expanding the Evaluator's Toolkit. The keynote speaker will be Leslie J. Cooksy, AEA's 2010 President and Founder and Director of the Interdisciplinary Graduate Program in Evaluation at the University of Delaware. Pre-conference workshops will be led by Susan Berkowitz, Westat, Qualitative Data Analysis with NVIVO and Joy Quill, CJ Quill & Associates, Effective Writing for Evaluators. A Sunday evening interactive and networking session will be led by John Kelley, Villanova University.

"This year's conference will be the best ever," says Jill Feldman, current EERS co-president, "for gaining new skills and knowledge in an intimate and welcoming atmosphere, engaging in cross-disciplinary dialogue, and rubbing elbows with national evaluation experts in an informal setting."

For best rates, register by March 1. For more information, visit the EERS website.

Go to the EERS Website

Many Ways to Contribute to AEA - As Silent Auction Coordinator, etc.

Looking for ways to get involved in the life of the association? AEA's new Member Involvement Initiative (MII) has the following updates related to new volunteer opportunities within AEA: 

Evaluation FAQs Working Group: AEA would like to develop short, straightforward answers to Frequently Asked Questions such as "What is evaluation, What does an evaluator do, and Why is evaluation useful?" This is intended for use on the AEA website, in press releases and in materials used internally but also externally with non-practitioners who might be potential users, partners or simply have an interest in better understanding evaluation and evaluators. Our working group will meet monthly by phone May-October with ongoing email exchange around draft language. We envision the participation of a small group of seasoned veterans, as well as newcomers to the fold. Those with expertise in communications are especially valuable in understanding the goals of the initiative, translating complex information for public consumption, and understanding and appreciating the nuances of the field. If you are interested in serving in this capacity, please contact AEA Communications Director Gwen Newman at [email protected]. (Deadline: March 15)

Evaluation 2010 Proposal Reviewers: We'll receive over 1000 proposals to present at Evaluation 2010! The proposals are reviewed by teams from AEA's 40+ Topical Interest Groups (TIGs). If you would like to be considered to be a conference proposal reviewer, please do either (but not both) of the following:

  1. If you are submitting a proposal, there is a space right on the proposal submission form to indicate that you would like to be considered to be a reviewer and to indicate the TIGs for which you are interest in reviewing; OR
  2. New! If you are not submitting a proposal, then send an email to Heidi in the AEA office at [email protected] with the subject line "AEA Proposal Reviewer" noting that you are interested in reviewing and identifying up to two TIGs for which you would like to be considered as a reviewer.

The list of TIGs may be found online here. Please note that taking either approach will result in your name being added to a list of potential reviewers that is supplied to the TIG Program Chairs. The TIG Program Chairs will contact potential reviewers at their discretion. Each TIG's review process is slightly different, but all conduct their review during the month of April. When/if a Program Chair reaches out to you, she or he will include information about the expectations and timeline and will reconfirm your interest and availability. (Deadline: March 19)

International Silect Auction Coordinator: Each year at the annual conference, the International and Cross-Cultural (ICCE) TIG hosts a silent auction, the revenue from which benefits a fund to assist evaluators from developing countries and countries in transition to attend the annual meeting. After many years of dedicated service, our previous auction coordinator is stepping down and we're seeking a new coordinator to fill her shoes. The coordinator (a) solicits donations prior to the event through postings to discussion lists, outreach to TIG members, and her or his own professional network, (b) maintains a list of committed donors, (c) sets up and oversees the auction on site at the conference with the help of volunteers, and (d) manages the checkout on site and hands off the total collected to AEA staff. The total time commitment is approximately 5 hours during the fall before the event and an additional 7 hours on site at the conference. Please email ICCE TIG Chair Tessie Catsambas at [email protected], if you are interested in serving in this capacity. (Deadline: April 1)

New Jobs & RFPs from the AEA Career Center  
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:

  • Evaluators for Institutional Development programme for W/C African National Societies at International Federation of Red Cross and Red Crescent Societies (Geneva, Switzerland) 
  • Research Analyst at The Improve Group (Mendota Heights, MN, USA) 
  • Evaluation Specialist at University of Texas Health Science Center at San Antonio (San Antonio, TX, USA)  
  • Researcher at OMNI Institute (Denver, CO, USA)
  • Research Associate at Pacific Research & Evaluation (Portland, OR, USA) 
  • Research Scientist at Alliances for Quality Education, Inc. (Largo, MD, USA) 
  • Health Education/Evaluation Specialist at Impact Assessment Inc. (Richmond, CA, USA)
  • Project Manager for International Evaluations at ICF International (Calverton, MD, USA)
  • Application Reviewer for Teaching American History Grant at US Department of Education (Washington, DC, USA)
  • Community Mental Health Improvement Advisor at National Center for Child Traumatic Stress, Evidence-based Practice Implementation Center (Durham, NC, USA)
Descriptions for each of these positions, and many others, are available in the AEA Online Career Center. According to Google analytics, the Career Center received close to 4000 unique visitors in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee.
 
Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed.
 
Go to the AEA Online Career Center
Get Involved
Get the most from your membership by taking advantage of the many things that you can do right now to participate in the life of the association, share your input, and promote your business.
About Us
The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
 
The American Evaluation Association's mission is to:
  • Improve evaluation practices and methods
  • Increase evaluation use
  • Promote evaluation as a profession and
  • Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275