News from The American Evaluation Association

Newsletter: June 2014

Vol 14, Issue 6

Message from the President: Evaluation Is a (Global) Family Affair

Dear AEA Colleagues,

First, let me thank those of you who sent your six-word stories in response to last month's column. As you recall, I asked members to write six-word stories about what this year's conference theme — Visionary Evaluation for a Sustainable, Equitable Future — means to you. The six-word requirement is a tribute to Ernest Hemingway, who, as legend has it, once won a bet by writing an entire story in just six words. We have received many fantastic submissions, and I encourage those of you who haven't yet submitted a story to send one to sixwordstory@eval.org. As we receive your stories, we'll look for ways to incorporate them into the conference.

I'm writing this message while en route to Ottawa to attend the Canadian Evaluation Society's (CES) annual conference. This has me reflecting on the longstanding relationship CES and AEA have, and why this is so important to AEA members.

To me, attending the CES conference is like attending a family reunion. Each year more and more people join the family, and it is always fun to meet them. It is also a pleasure to reunite with those who have been "in the family" for a long time. Such is the relationship between CES and AEA — an extended family, and one we have been a part of for a long time.

We care about growing our international evaluation family — after all, there is strength in numbers — as well as bringing this family together. Fortunately, we have ways to do this.

The International Organization for Cooperation in Evaluation (IOCE), established in 2003, is the international connector for our global evaluation family. It is a partnership of national and regional evaluation associations, and it provides leadership in evaluation worldwide. IOCE encourages demand for evaluation and supports organizational capacity for evaluation associations.

Voluntary Organizations for Professional Evaluation are avenues for connection. VOPE is an umbrella term for all kinds of groups interested in advancing evaluation as a profession. VOPEs can function at regional and national levels, and they bring together evaluation practitioners from various fields, including government, academia, and the private sector. The number of regional, national, and international VOPEs grew from just 15 in 1999 to more than 188 by the end of 2013.

AEA has also established an International Working Group headed by Hubert Paulmer. Among other things, this group is establishing a Partnership Protocol that will outline ways that AEA can partner with international VOPEs throughout the year. AEA is also working on ways to support and highlight the International Year of Evaluation 2015, including its participation in EvalPartners. This Working Group is also exploring ways to connect on conversations related to international topics through virtual coffee break type formats. (More news on these topics to come.)

The number of regional, national, and international VOPEs grew from just 15 in 1999 to more than 188 by the end of 2013. We are indeed a large family, and we have much to learn from one another and many opportunities to contribute to the public good.

As a family, we have a great responsibility to support the well-being of society within our own countries and globally. So many issues we face today are not bound by national boundaries. As we see the globalization of markets, environmental issues, conflicts, and innovation, we realize just how important it is that we unite with other evaluators worldwide to engage with these issues. I encourage all of you to consider how you can take a global approach to your own work. It may range from working on international evaluations to going online to learn about other evaluation associations throughout the world. Together we can support a more equitable and healthy world.

Warm regards,

Beverly Parsons

AEA 2014 President

In This Issue

Quick Links

AEA Guiding Principles

Topical Interest Groups

Subscribe to AEA 365

Affiliates

Board

Important Note

To ensure this newsletter reaches you every month, add info@eval.org to your email contacts!

Join our Mailing List!

AEA Values - Walking the Talk with Nicole Robinson

Are you familiar with AEA's values statement? What do these values mean to you in your service to AEA and in your own professional work? Each month, we'll be asking a member of the AEA community to contribute her or his own reflections on the association's values.

AEA's Values Statement

The American Evaluation Association values excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community.

i. We value high quality, ethically defensible, culturally responsive evaluation practices that lead to effective and humane organizations and ultimately to the enhancement of the public good.

ii. We value high quality, ethically defensible, culturally responsive evaluation practices that contribute to decision-making processes, program improvement, and policy formulation.

iii. We value a global and international evaluation community and understanding of evaluation practices.

iv. We value the continual development of evaluation professionals and the development of evaluators from under-represented groups.

v. We value inclusiveness and diversity, welcoming members at any point in their career, from any context, and representing a range of thought and approaches.

vi. We value efficient, effective, responsive, transparent, and socially responsible association operations.

See AEA's Mission, Vision, Values

Nicole Robinson is an independent evaluation consultant and doctoral student in Milwaukee.

I truly believe evaluation is values oriented and that it can support liberation and equality. I say this because values have been part of my personal and professional life. I grew up in a house and in a family with strong values, where values were spoken, modeled, and enforced. My parents, grandparents, and relatives didn't refer to their values as progressive or social justice — the labels didn't matter. They worked hard, treated people and the environment with respect, and considered the larger community and future generations in their everyday lives and choices. When I left the nest, my first job out of graduate school was with a values-based nonprofit organization. Like my family, my employers embodied their values in every thought, action, and decision they made. Those values were also enforced!

A huge shock came when I moved back to my hometown after several years. The inequalities there were so apparent and tangible. Every day was uncomfortable. The inequality was uncomfortable. Being in multiple environments absent of strong values (or even basic ones like fairness) was especially uncomfortable. Even more, when I saw that few decision-making tables included people with my shade of black and brown and that too few of the evaluators providing data and expert opinion had similar backgrounds as mine, I knew something had to be done — but that was as far as I could think. I couldn't think of a solution because I was hurting and overwhelmed by how bad things were. Where were the values?

Going to AEA's Conference on Values and Valuing changed everything. (This isn't propaganda. The conference really was a catalyst.) I met key people and received validation that evaluation has values, that evaluators can be diverse, and that the field needs diverse evaluators. After that, several colleagues and I started on an amazing journey. We began thinking about AEA's cultural competency standards and are still working now to identify tools, rubrics, and litmus tests that would hold evaluators accountable to the highest degree of those standards. This work is still in progress and is a core part of ¡Milwaukee Evaluation! Inc.'s field-building activities. We are also working on increasing the number of evaluators of color and the number of pathways for evaluators of color and other underrepresented groups to enter and stay in the field. We bit off a lot, but to make real progress we have to be bold and enforce our values!

Face of AEA - Meet Danielle Murillo

Name: Danielle Murillo

Affiliation: California State University, Dominguez Hills

Degrees: B.A. in psychology, California State Polytechnic University, Ponoma; M.A. candidate in sociology, California State University, Dominguez Hills; Certificate in grant writing and administration, April 2014

Years in Evaluation Field: 3.5

Joined AEA: 2013

Why do you belong to AEA?

I joined AEA to learn more about the field of evaluation while expanding my professional network.

Why do you choose to work in the field of evaluation?

I am interested in evaluation because it allows me to combine different aspects of my training in social science, which is research methodology and the helping professions, thus applying it to real settings. Evaluation seeks to answer if a program, policy, etc. is doing what it initially intended to do and is serving the population of interest to the best degree possible. An evaluation will provide data-driven results to help stakeholders and funders determine if a program should be improved, terminated, or a new one should be implemented based upon the needs of the people. It is a method used to improve the human condition and help people in the best form possible.

What's the most memorable or meaningful evaluation that you have been part of?

The most memorable evaluation that I have been a part of was the Achieving Our Mission at Cal Poly Pomona in 2008. Near the end of my college career, I enrolled in a program evaluation class. The class assignment was to work with Cal Poly Pomona's Western Association Scholastic Committee (WASC) and evaluate how Cal State Northridge became a learning-centered institution. During this process, which involved transcribing the interview with Cal State Northridge, weekly meetings with a member from the WASC panel, and presenting the results to the class, I learned the importance and amount of effort required in program evaluation. Upon completing the course, I began to question how I could make a program more effective so that it meets the client's needs.

I really enjoyed the research process while in college and initially hoped to pursue graduate school in experimental psychology. However, after completing this course in program evaluation, I fell in love with the field and wanted to learn as much as I possibly could.

What advice would you give to those new to the field?

I am still learning and am relatively new to the field, but I would recommend the following:

Join a professional organization such as AEA or LinkedIn and attend conferences
View AEA's Coffee Break Discussion Series, which explores different aspects of evaluation
Try to gain evaluation experience — it is the best teacher because you "learn by doing"
Read literature within or pertaining to the field. Some of the books/materials that have helped me are:

Evaluation Essentials A to Z by Marvin Alkin
The Art of The Case Study by Robert Stake
Qualitative Research and Evaluation Methods by Michael Patton

Attend professional development workshops or training such as those hosted by Claremont Graduate University

AEA's more than 7,800 members worldwide represent a range of backgrounds, specialties, and interest areas. Each month, we profile a different member via a short question-and-answer exchange. If you would like to be featured in a future Face of AEA profile, send an email to eschmitt@eval.org.

Diversity - The Graduate(s): The 10th GEDI Cohort Graduates at Summer Institute

The Graduate Education Diversity Internship (GEDI) graduation at Summer Institute 2014 marked 10 years of GEDI excellence! Continuing the legacy of diversity at AEA, the GEDI program engages and supports students from groups traditionally underrepresented in the field of evaluation. Crystal Coker, Shipi Kankane, Bailey Murph, and Anael Ngando made their final presentations of the internship during an intimate luncheon attended by special guests, including AEA President Beverly Parsons.

A mainstay at the institute, the GEDI graduation celebrates the successful completion of all internship program requirements by the current cohort. GEDI interns spend the academic year actively participating in specially designed training in Culturally Responsive Evaluation. These trainings culminate in related deliverables due to exercise and applied learned concepts. Joining the alumni ranks of the 50 successful GEDI graduates, this year's impressive cohort is eager to impart their expertise on the world of evaluation.

At the helm of this year's cohort are program directors Dr. Stewart Donaldson and Dr. Ashaki Jackson, as well as program liaison Dr. John Lavelle. "Our 10th GEDI cohort anniversary maintained the high caliber of scholarship to which this program has become accustomed," says Dr. Ashaki Jackson, rounding out her second year as a GEDI program co-director. "During their academic year, Anael, Shipi, Crystal, and Bailey worked seamlessly to complete useful deliverables that can enhance AEA outreach efforts. They also facilitated improved understanding of cultural competence within their internship sites and implemented individual projects to bolster evaluative thinking, all the while accepting the challenges naturally associated with these tasks. We were pleased to see that this cohort was a comfortable and close one where long-term friendships were made. We commend them on their hard work."

AEA congratulates the Anniversary GEDI Cohort! We couldn't be prouder of their hard work and are expecting spectacular things from them in the near future. Learn more about the GEDI program here, and get to know this year's cohort here. AEA and the GEDI program directors are now in the process of reviewing applications for the next set of future evaluators. Stay tuned for the announcement of the next cohort as they set out on their GEDI journey!

Congratulations to the 2013-2014 GEDI Cohort!

Crystal Coker, Center for Elementary Mathematics and Science Education, Chicago
Shipi Kankane, University of Minnesota, Extension Center for Youth Development, St. Paul, Minn.
Bailey Murph, IMPACT Planning Council, Milwaukee
Anael Ngando, Education Development Center, Washington, D.C.

Policy Watch - One Person Can Help Change the World ... of Evaluation!

From Cheryl Oros, Consultant to the Evaluation Policy Task Force (EPTF)

Over the past year and a half many AEA members have offered assistance in developing and promoting evaluation policies; building policymakers' interest in the results of their evaluations; encouraging legislators or agencies to provide sufficient resources for quality evaluations; and raising the quality of evaluation work in the field. Some members have also engaged me in discussions about their own organization's evaluation policies, the influence AEA's Roadmap has had, and their needs for additional policy guidance from AEA.

What constitutes evaluation policy, and why is it important? Evaluation policy has to do with rules, whether formal or informal, that an organization establishes for planning, conducting, and/or using evaluation. Evaluation policies can address such things as requirements, funding, methods, dissemination, and quality assurance for evaluation. Evaluation policies can be promulgated through laws, regulations, administrative procedures, budgets, organizations, standards, and even informal norms. At the federal level, they may be established by Congress or executive agencies. Evaluation policies can also be promulgated by state and local governments, foundations, other countries, or any organization seeking to make evaluation part of the way it does business. In 2007, AEA established the EPTF to promote efficacious evaluation policies.

How can you help promote effective evaluation policies?

Alert the EPTF about policy-influencing opportunities. For example, contacts from outside the EPTF led it to provide input on legislation that requires evaluation of foreign assistance programs; publish an analysis of impact evaluation requirements of the Federal Program Assessment and Rating Tool; provide written testimony on requirements for evaluation of poverty reduction programs for families; and provide technical assistance to Congressional staff on funding an evaluation of the President's Emergency Relief Program for AIDS. Likewise, contact from AEA members led the EPTF to offer advice to the Office of Management and Budget about the Paperwork Reduction Act and its effect on evaluators' work. Based on these and other examples, I encourage you not to hesitate to alert the EPTF about opportunities to influence evaluation policy!
Share your organization or country's evaluation policies and related experiences with the EPTF. Especially with the EPTF's expanded focus on state and international areas, such information would be useful.
Provide copies of AEA's positions about evaluation policy to policy makers (such as Legislative and Executive Branch staff). The Evaluation Roadmap and other brief policy position leaflets are available on the AEA website. The August 2013 Policy Watch column described the joint initiative of the Washington Evaluators and EPTF to encourage AEA members attending the 2013 conference to visit Congress/Congressional staff to introduce AEA and its policy guidance (and to establish valuable contacts).
Provide input during the AEA Board of Directors approval process of any new EPTF draft policy documents. The board approved a policy to facilitate member involvement under a variety of circumstances, including long-term efforts to produce carefully vetted position papers and shorter-term policy opportunities, such as comments on proposed government regulations.

You can visit the EPTF website to see the kind of public evaluation positions that AEA takes. The EPTF will soon have an updated, more user-friendly site with materials for both AEA members and evaluation stakeholders. You can also reach the Evaluation Policy Task Force via email at EvaluationPolicy@eval.org.

Potent Presentations Initiative - Conference Prep Plan of Attack

From Stephanie Evergreen, Potent Presentations Initiative Coordinator

In a week or so you will get the news about your conference proposal. After being accepted to speak at a conference, most people prepare in this way:

Forward acceptance email to co-presenters
High five
Do nothing until September
In September, SCRAMBLE at top speed to pull together some decent material

This does not have to be your plan of attack. Preparing for a top-notch conference presentation is more like a marathon, not a horse race. This year, distribute your efforts over the months leading up to the conference so that you can actually enjoy your time there, learning from your colleagues, rather than holing up in your hotel room, stitching together your presentation. This year, try this:

Head to p2i.eval.org and download the Presentation Preparation Checklist. (Go ahead and download it now. It's hugely helpful for all kinds of presentations, not just those at our annual conference.)
Forward acceptance email to co-presenters, with checklist attached.
High five.
Bask in your awesomeness for a day or so and then email your co-presenters to request a 30-minute meeting to review the first set of bullet points on the checklist, divvying up tasks. These include:
- Choose 1-3 key content points to be conveyed and then develop notes regarding what you wish to share relating to each key point.
- Gather photos or images for use in slideshow.
- Check in with co-presenters on timeline.
- Expect to hear from your session chair by email.
  - Ask about length of time for your presentation, discussion time to be reserved for audience questions and a discussant, and the sequence of those events during your session. Papers have about 15 minutes. If you are part of a panel, demonstration, think tank, etc., determine with your chair and co-presenters how much time is to be devoted to what content.
  - Ask about your colleagues' presentations and coordinate content to limit overlap and respond to one another's work.

If you can manage this much, you will be leaps and bounds ahead of the typical schedule, for a healthier, calmer, more prepared plan of attack.

Nominees Sought for AEA Representative to COSSA for 2015-2017 Term

AEA is accepting applications to become the AEA representative to the Consortium of Social Science Association (COSSA). The deadline to apply is Friday, August 1, 2014.

AEA joined COSSA in 2011 to enable us to work with other social sciences in an organized fashion to communicate with federal officials. At that time, George Grob, the co-chair of AEA's Evaluation Policy Task Force (EPTF), said "AEA's making common cause with COSSA makes us both stronger. The joining of our voices will enhance public support for what we do and increase the odds that policymakers will heed the advice of scientists like us. At the same time, we stand to learn a lot from our comrades in related professional disciplines."

COSSA was founded in 1981 and today over 100 professional associations, scientific societies, and the like belong to COSSA. With a full-time professional staff, it promotes federal funding for research in the social and behavioral sciences and communicates findings of the sciences to the Washington policy-making community. It also makes its members aware of federal policies relevant to the social and behavioral sciences. COSSA's work is primarily concerned with basic social science research, but our representative might choose to affiliate with other members whose fields are more applied to develop an applied subgroup within COSSA.

AEA's representative to COSSA receives biweekly newsletters from COSSA on its work and communicates information pertinent to the evaluation field to AEA leaders. He or she also represents AEA at an annual COSSA conference, and AEA pays the expenses for that conference. The COSSA representative submits a written report to the Board on COSSA and its evaluation-related activities each year. The representative will serve a three-year term beginning January 1, 2015, and ending December 31, 2017.

The person serving in this position is likely a senior member of the evaluation profession, dedicated to the field, and has the breadth and depth of knowledge and practice required to speak authoritatively about evaluation issues in a multidisciplinary context. Individuals seeking to serve need to have a demonstrated range of relevant experience in evaluation and a commitment to representing the diversity of people and disciplines represented in the field and in AEA's membership.

To learn more about the work of the COSSA, please review the COSSA website. Our past representative, Jody Fitzpatrick, is available for inquiries if you have specific questions regarding the position. She may be reached at jody.fitzpatrick@ucdenver.edu.

If you are interested in being considered as AEA's representative to COSSA, please compile the following into a single file and submit by Friday, August 1, 2014:

A one-page statement of interest indicating why you would like to serve;
A one-page bio describing your evaluation background and your experience in interdisciplinary settings and working with committees or organizations;
A one-page letter of support for your nomination from another AEA member; and
A current curriculum vitae or résumé.

Please send your submission as a single file via email to the AEA office at info@eval.org. The selection of the next COSSA representative will be made by the AEA president and president-elect, in consultation with the AEA Board.

If you have questions, please do not hesitate to contact AEA President Beverly Parsons at president@eval.org.

Save the Date: AEA's Evaluation 2014, Oct. 15-18, in Denver

Evaluation 2014, the 28th Annual Conference of the American Evaluation Association, will be held at the Colorado Convention Center and the Hyatt Regency on Oct. 13-19, 2014. Housing and registration information will be available in mid-July.

Find out more!

AEA Needs You! Volunteer Opportunities with AEA

AEA is recruiting volunteers to work on specific projects in the next few months. Some of these projects involve working directly with staff, and others are in a working group format.

Below are the current volunteer needs. If you are interested in serving in some capacity, please contact the emails listed below. AEA values diversity and seeks volunteer representatives with a variety of backgrounds, skills, perspectives, and demographics. See AEA's Values Statement.

Videographer: AEA needs several videographers for upcoming projects related to the Annual Conference. Each project could require up to 20 hours of work from July to September. One videographer will work with a working group on a project related to the Six-Word Story project. Please contact Zgrays@eval.org with your interest and a description of the kind of experience you have with these types of projects. Please also indicate the video format and software program you use.
Professional Development Working Group: The volunteers in this working group will assist staff in creating the professional development strategy for AEA's e-learning programs. The work begins in July and runs through the end of 2015. Please contact Llawson@eval.org to express your interest.
Social Media Working Group: The volunteers in this working group will assist staff in developing AEA's social media strategy and ways to better engage with evaluators and the evaluation community on social media. Volunteers should be currently engaged in social media activities and familiar with the various ways that AEA is engaging the evaluation community virtually. The work begins in late July and will run for at least a year. Please contact dmcdonnel@eval.org.

Stay tuned for additional volunteer opportunities being announced in the coming months. We hope to hear from you!

eLearning Update - Discover eStudy Courses

Our eStudy program is made up of in-depth virtual professional development courses. Below are July's eStudy offerings:

eStudy 044: Getting Started: Introductory Consulting skills for Evaluators - Gail Barrington
July 9, July 14, July 16, and July 21

2-3:30 p.m. ET

Are you a program evaluator contemplating venturing out on your own? For many, this may be both an exciting but intimidating prospect. Taught by an independent consultant, this practical eStudy course will reveal the simple but important skills needed to be successful. The course helps participants apply management consulting, entrepreneurial, and small business skills to the evaluation and applied research setting. Valuable samples, worksheets, insider tips, and trade secrets are provided. Through lecture, anecdote, at-home exercises, and independent reflection, participants are encouraged to identify and address the unique issues they face and to prepare an action plan.