Newsletter: October 2010Vol 10, Issue 10

Banner600

 

cooksy2Dear Colleagues,


We announced the recipients of this year's 2010 AEA awards in our last newsletter and I want to publicly congratulate them all for their significant accomplishments and the honor that this represents for each of them individually and for all of us collectively. My best to Beth Weitzman, Jim Rugh, Jonathan Morell and Rodney Hopson, recipients of AEA's Outstanding Evaluation Award, Evaluation Practice Award, Evaluation Theory Award and Service Award. Their work is inspiring, and I cannot wait to see them - and you - at this year's conference.

A final announcement: Actress Anna Deavere Smith, who I was so excited to have kick-off the conference, will be unable to participate. And while we will miss her, I am very pleased to announce that a panel of AEA luminaries - consisting of Eleanor Chelimsky, Laura Leviton, and Michael Patton - will take her place with a lively discussion of timely relevance. The plenary will have a different orientation, with a diversity of views on current issues of interest to the U.S., the world and especially our field. Join us as they share their unique thoughts on Tensions Among Evaluative Perspectives in the Age of Obama: Influences on Evaluation Quality, Thinking and Values.

 

The countdown is on and I look forward to seeing many of you in San Antonio!

 

Leslie

Leslie Cooksy
2010 AEA President
In This Issue
Policy Watch with George Grob
TechTalk with LaMarcus Bolton
NSF Funding Opportunities
CES Credentialing Update
AEA In the News
Nights Out in San Antonio
Client Communications - IPDET
Book: The Networked Nonprofit
Volunteer Opportunities
New Jobs Postings
Get Involved
About Us
Quick Links
Policy Watch - Evaluation Roadmap: It's Official!
From George Grob, Consultant to the Evaluation Policy Task Force

Grob

Max Ehrmann, in his poem Desiderata, advised his son to "Enjoy your achievements as well as your plans." That is good advice for all of us now with the approval by AEA members of An Evaluation Roadmap for a More Effective Government.

 

Two years ago this month, the Evaluation Policy Task Force (EPTF) began an intensive campaign to connect with the incoming administration of President Obama about the importance of evaluation. The Roadmap emerged from the initial papers and interactions with Office of Management and Budget OMB) staff.

 

This document is worth celebrating for several reasons. First and foremost is its content - a digested but still detailed statement about the need for evaluation as "the DNA" of federal programs, as outgoing OMB Director Peter Orszag called it, and a set of principles to guide adoption of effective evaluation practices within the federal government. The Roadmap's themes can be seen in the evaluation policies of the Obama administration, which we have described in several "Policy Watch" articles.

 

Equally important is the process whereby this document was crafted. AEA has long struggled to produce policy statements that are both timely and reflective of its memberships' views. With the Roadmap we found a way forward. This version reflects comments provided by interested members through an open call for comments and was not finalized until its approval this month by an open membership vote. The process that emerged during the Roadmap development has been adapted to the formulation of comments on the Paperwork Reduction Act; the Maternal, Infant, and Early Childhood Home Visiting program; and the Government Auditing Standards. We are now working to develop final policy for the Board to review at its January meeting.

 

So what's next? First and foremost, we urge all AEA members to read the Roadmap and to distribute it broadly through any of their own professional networks that are tied to the federal government. The more who know about it, the better.

 

Beyond that, there is much more evaluation policy to develop, publish, and implement. The current Roadmap should be regarded as "version 1." It is a living document that can be updated periodically to reflect both the needs of government policy makers and the maturation of the evaluation field itself. There may also be needs and opportunities to develop official positions on other policy matters. For example, one member has suggested developing a statement on the practical aspects of promoting evaluator independence within government agencies.

 

The EPTF encourages AEA members to identify policy areas of critical concern to evaluators. The evaluation policy discussion list provides a mechanism to exchange ideas on such matters and I encourage you to subscribe via the EPTF homepage.

 

Go to the EPTF Homepage to view the Roadmap or Join the Discussion List

TechTalk - Leveraging Technology in Evaluands
From LaMarcus Bolton, AEA Technology Director
Bolton
In the previous TechTalk article, I described how mobile smartphone technology could be used to increase productivity and overall workflow. When I sat down and spoke with various 2010 AEA/CDC Summer Institute attendees, I discovered how other technological tools were used for evaluation-related tasks and projects.

Web-based survey applications are a popular choice for collecting data, as there are tons of great, easy-to-use options available. Amongst popular options mentioned are Zoomerang and Qualtrics. However, several attendees felt that Survey Monkey was the simplest and most user-friendly option. In terms of use, Stacy Carruth, Community Health Specialist from the Regional Center for Healthy Communities, noted that Survey Monkey is used, "... [to] collect data for our training needs assessment and help coalitions evaluate their work."

Additionally, many of the attendees feel that statistical applications are imperative for analyzing collected data. David Urias, the Director of the Evaluation Research Network and Assistant Professor of the International Education and Program Evaluation Department at Drexel University, integrates SPSS into his workflow. Conversely, Steve Fleming, Senior Systems Analyst of the National Center for Educational Achievement, prefers SAS. Steve noted, "[SAS] allows me to manipulate data in almost any way I like, fit most statistical models, and create decent quality graphs."

Technology is also used for alternative forms of data. Heather Clark, Evaluation Manager at the Center for Community Health Development at Texas A&M, uses UCINET for social network analysis. Using this program, Heather is able to study community groups, including: determining who is connected to whom, who is sharing what resources, and examining how groups have changed over time. Additionally, Charlotte Kabore, CDC Public Health Advisor at the Oklahoma State Department of Health, uses GIS technology to analyze her data while focusing on intervention. With GIS, she is able to, "...take data down to the local level" and see maps of where she needs to focus her efforts.

When it comes to evaluands, effectively feeding back data is crucial to success of the project. Rebecca Buzzard, Program Coordinator and Evaluator at Craven Smart Start, describes how she uses Excel to present the results of her projects: "I use the formulas to help with calculations and graphs for images. I also use pivot tables." Wordle, the web-based tool used to create "word clouds," is widely used for presentation as well. Felix Blumhardt, Lead Evaluator for the Evaluation Group, explained, "I use Wordle as a kickoff of snapshots or themes that are present in focus groups, blogs, and other text. I use it as emphasis after charts and text." Furthermore, Carruth believes that Wordle helped her organization think more creatively about data presentation overall.

This article is one part of a short series on technology use in evaluation. If you have interesting ways of using technology in your work, please consider sharing within our technology forum.
If you have any other questions, I can always be reached at [email protected].
New Funding Opportunities from NSF for Research on Evaluation

The National Science Foundation (NSF) has announced new funding opportunities that will impact research on evaluation. NSF's Education & Human Resources Division (EHR) calls the initiative Promoting Research and Innovation in Methodologies for Evaluation (PRIME). The PRIME program seeks to support research on evaluation with special emphasis on exploring innovative and new approaches for determining the impacts of STEM (Science, Technology, Engineering and Mathematics) education projects and programs, building on and extending the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and growing the capacity and infrastructure of the evaluation field. The PRIME  program seeks to advance evaluation theory and practice across all levels of the STEM education enterprise in both formal and informal settings. Two types of proposals will be supported by the program: Exploratory Projects that include proof-of-concept and feasibility studies and more extensive Full-Scale projects. The proposal due date is January 5, 2011.

 

NSFThe PRIME solicitation builds on prior programs and coincides with increased federal emphasis on evaluation and in the interest of furthering the development of new approaches and methods to evaluate STEM education programs.

 

"This solicitation is informed by new advancements in STEM education and seeks proposals that bring innovative evaluation approaches and methodologies to the advancement of the STEM education enterprise," says NSF's Leslie Goodyear. "There are many researchers and evaluators out in the field doing innovative work in evaluation and we'd like to see them focus on STEM education by bringing new conceptual frames, methods, and measures to the evaluations of these critical programs."

 

"Evaluators and education researchers who focus on STEM education will likely be most interested, but we're also encouraging the "unusual" suspects, such as social science researchers, policy researchers, or STEM researchers, to propose new and innovative approaches that could be used in STEM education evaluation. This is an opportunity to develop new theories of evaluation, bring methodologies from other disciplines to STEM education evaluation, and to build the capacity of the evaluation and education research communities to conduct evaluation of STEM education programs and projects. It is also an opportunity for researchers, evaluators and STEM discipline experts (engineers, scientists, mathematicians, etc.) to work together to inform how STEM education programs are evaluated."

 

NSF representatives will be on hand at this year's Evaluation 2010 conference in San Antonio. If you have questions you can enquire onsite, visit the PRIME site, or email [email protected].

An Invitation and Update from CES - Learn More About Credentialing Program
From Martha McGuire, President of the Canadian Evaluation Society

McGuireThis is one of the most exciting years to head up the Canadian Evaluation Society (CES). First, CES is focusing on its Professional Designation Program (PDP), which was launched at our Annual Conference in May and is generating quite a bit of interest in the field. This is a major initiative that is getting world-wide attention. It presents a number of challenges: 
  • Developing CES' capacity to receive and process applications from those seeking the Credentialed Evaluator designation;
  • Ensuring a fair and equitable process so that all applicants receive the same consideration; and
  • Creating a balanced process so that the designation is meaningful and at the same time does not exclude excellent evaluators.
Keiko Kuji-Shikatani, the Vice President responsible for the PDP, along with members of the Credentialing Board, have been working diligently to ensure inter-rater reliability. There clearly is an interest based on early response and, not surprisingly, some concerns. Some senior evaluators have expressed concern that the process is too onerous. It can take as long a full day to complete the application form and provide all of the supporting evidence. On the other hand, the consultation process leading up to the establishment of the program, indicated that the CES membership wanted a meaningful designation. All and all, the process is moving along quite smoothly with applications continuing to arrive.

We have also been presented with a major issue in Canada. The federal government has abolished the mandatory Census Long Form and is replacing it with a voluntary census. This was a primary source of information for program planning as well as for monitoring demographic and economic changes over time. The Federation of Francophone and Acadian Communities of Canada has asked the court in Ottawa to void the new policy and wants an injunction that would keep the new voluntary national household survey from being distributed this year. The federation, which has also filed a complaint with Canada's official languages commissioner about the census change, argues that the government's new policy violates not only the Official Languages Act, but also the Charter of Rights and Freedoms. It is a major issue, of serious concern to planners and evaluators, alike. The Chief Statistician of Canada resigned because he so strongly disagreed with this move and CES responded with a letter of protest and is now considering next steps.   

In the meantime, we invite you to learn more about CES' credentialing program. There will be a special session at AEA's conference on Thursday, November 11 at 3:35 p.m. in the Travis C room.  I would encourage anyone who is interested to attend. AEA members who are also CES members are eligible to apply to become Credentialed Evaluators. And, at this point, it is the only credentialing program in the world. I look forward to seeing you soon!

Go to the CES Credentialing Website
In the News
BickmanCongrats!
AEA member Leonard Bickman was recently awarded a Fulbright Senior Scholarship to develop a large-scale demonstration project to explore the best ways to provide and evaluate mental health services to youth through e-therapy. The project will combine his team's work on web-based feedback to clinicians with web-based therapeutic programs specifically for youths. Bickman is a professor at Vanderbilt University in Nashville, Tennessee.

MarczakUniversity of Minnesota's Mary Marczak, an extension evaluation and research specialist, has been honored with a Distinguished Leadership Award. Mary works alongside Extension teams to evaluate more than 70 family and youth programs, ensuring that programs make a positive difference in people's lives. Marczak has served on a National Evaluation Collaboration Project funded by the USDA Children, Youth and Families at Risk (CYFAR) National Initiative and served as both national program chair and chair of AEA's Extension Education Evaluation Topical Interest Group.

HyattDid You Know?
San Antonio's Grand Hyatt is really going green? The hotel recently planted its own organic garden, with more than 200 newly planted herbs including chives, mint, oregano, rosemary, and sage. The herbs are used in the more than 1,500 meals the hotel serves daily. Check it out at this year's conference!
Are You Ready - Thursday Nights Out Events at Evaluation 2010 Announced
While in San Antonio for this year's Evaluation 2010, check out some of the local favorites in dining and entertainment. And, for those who'd like to connect with others who share similar interest, you can sign up now for AEA's special Thursday evening Nights Out events that take participants to a variety of venues featuring food and fun. For your convenience, we provide a sneak preview of this year's line-up below. In all, there are 17 outings from which to choose. So look over your options, reserve your spot, then meet your group for an evening out. It's that simple and a lot of fun. Thanks to all hosts who volunteered to lead their small groups to and from their destinations and be part of a memorable evening of casual connecting with friends and colleagues. This is an annual activity that gets rave reviews. Enjoy!
2010 Nights Out Event Options
  • Azuca
  • Boudro's Full!
  • Chart House Restaurant
  • Fig Tree Restaurant
  • Green Vegetarian Cuisine & Coffee Full!
  • Guadalajara Grill Full!
  • Iron Cactus Full!
  • Pat O'Brien's San Antonio
  • Schilo's
  • Zinc Bistro & Wine Bar
  • Zuni Grill
  • Howl at the Moon Full!
  • Liberty Bar
  • The Landing Full!
  • Dinner and Dancing at the Republic of Texas Restaurant & Nightclub Full!
  • Dinner and Laughs at Rivercenter Comedy Club Full!
  • San Antonio Ghost Tour
Client Communications - IPDET Newsletter
How do you communicate with your clients? AEA would like to spotlight samples of great client and stakeholder communications. Here, we feature a newsletter by the International Program for Development Evaluation Training (IPDET). To see the newsletter, click here.
IPDETIPDET is an international executive training program that aims to provide managers and practitioners the generic tools required to evaluate development policies, programs and projects at the local, national, regional, and global levels. An initiative of the World Bank's Independent Evaluation Group and Carleton University in Ottawa, Canada, the program strives not only to enhance skills and knowledge, but create networks.
"The newsletter has allowed us to convey important information to the IPDET and broader evaluation communities and to spotlight alumni achievements," notes editor Paul Glover. "It is traditionally circulated on the IPDET Listserv and posted on our website."
The IPDET newsletter was launched in November 2008 and is produced three times a year. It was created as a way to promote its upcoming events and important deadlines, distinctions and program specifics. The newsletter also provides a way for program participants to remain connected both before and after participation. Although it is occasionally used in print form (e.g. at major evaluation conferences), it was conceived primarily as an electronic document, so it needs to be short, visually attractive and point to the website or other references for more information.
  • The fall edition typically focuses on the previous program's participant evaluations; shares what's being planned for the next year; and announces registration and scholarship application dates for the upcoming program.
  • The winter edition is released as registration opens on the IPDET website and highlights new workshops and faculty as well as application deadlines.
  • The summer edition is distributed post-program and spotlights the successes and special moments of that year's effort, showcases recent IPDET graduates, and thanks scholarship sponsors for their support.
IPDET continually strives to foster communication within its community. In addition to a moderated listserv, an interactive website and some new social media, the IPDET newsletter offers yet another platform to nurture their network of participants. If you would like to be added to the electronic mailing list for copies of the IPDET newsletter, please send an email to [email protected].
 
To share samples of how you interact with your communities, email AEA's Communications Director Gwen Newman at [email protected]. We'd love to share the ways you communicate via this column as well as AEA's online eLibrary. Thanks!
The Networked Nonprofit: Connecting with Social Media to Drive Change

FineAEA member Allison Fine is co-author of a new title by Jossey-Bass that merges the worlds of ever-changing technology with the never-changing needs of the non-profit community. The Networked Nonprofit: Connecting with Social Media to Drive Change offers real-life scenarios from organizations that have successfully made the transition and shows readers how they can, too.

 

From the publisher's site:

"This groundbreaking book shows non-profits a new way of operating in our increasingly connected world: a networked approach enabled by social technologies, where connections are leveraged to increase impact in effective ways that drive change for the betterment of our society and planet."

 

From the author:

"There was a need for a book like The Networked Nonprofit to try to help nonprofit executives make sense of the cacophony of social media tools," says Fine. "The field needed a framework for understanding and using social media to enable organizations to take advantage of this particular moment in time.  It was great to be able to provide a broad overview of where we are right now, what's important and what's changing, with real case studies from nonprofits. The most exciting aspect of the book, I think, is that there are nonprofits that are born social, like MomsRising, and others, like Planned Parenthood, but are becoming Networked Nonprofits. The book is divided into two sections. The first part is dedicated to a way of being and the second to a way of doing. I think that structure, and the emphasis on being a certain way (e.g. open, transparent, focused on relationships not just transactions) sets it apart. Plus the real-life stories of organizations making that transition enables others to imagine making that change themselves."

 

About the author:

Allison Fine is a nationally-recognized writer and activist dedicated to understanding and enhancing efforts to use new social media tools for social change. Her research and writing focuses on the intersection of social media and social change. She is a frequent contributor to the Huffington Post, Personal Democracy Forum, Alternet and the Chronicle of Philanthropy and her articles have been published in the Boston Globe, San Jose Mercury Times and the San Francisco Chronicle. Founder of Innovation Network, Inc. (InnoNet), Fine also is the author of Momentum: Igniting Social Change in the Connected Age, which was lauded by the Wall Street Journal and the San Francisco Chronicle, and co-editor of Rebooting America.

 

AEA members receive 20% off the retail price of all books and journals ordered directly from Jossey-Bass as part of AEA's Publishing Partners program. To receive your 20% discount, use the promotional code "AEA10" online or call 1-877-762-2974. Email the AEA office first if you are ordering from outside the U.S.

 

Go the Publisher's Site

Volunteer Opportunities - AEA Newsletter Proofers
Looking for ways to get involved in the life of the association? AEA's new Member Involvement Initiative (MII) has the following updates related to new volunteer opportunities within AEA.

AEA Newsletter Sneak Previewers - AEA produces a monthly newsletter that features 8-12 short articles that include a combination of standing features and unique contributions from different member groups. The newsletter is deployed by email to AEA's more than 6,000 members and is also archived on AEA's website for future retrieval. In continuing efforts to improve newsletter quality and consistency as well as vital member input, AEA is looking for volunteers who would agree to proof each monthly issue immediately prior to its deployment for any last final edits due to typesetting changes, editing oversights, or variations in user platforms and newsletter appearance in format, fonts, spacing, etc. The perks: A sneak preview of each month's issue. The time required: the time it takes to read each issue (most likely 10-15 minutes beginning to end and to check for any obvious or more subtle variations in copy, links, formatting, etc.). The end result: A product that's consistent reader to reader regardless of email reader. If you are interested in serving in this capacity, please contact Gwen Newman in the AEA office at [email protected]. Deadline is Friday, November 19. Service is through 2011.

New Jobs & RFPs from AEA's Career Center  
What's new this month in the AEA Online Career Center? The following positions and Requests for Proposals (RFPs) have been added recently:

  • Branch Chief at USDA's Economic Research Service Market and Trade Economics Division, Field Crops Branch (Washington, DC, USA) 
  • Project Manager at VisionTech360 (Spokane, WA, USA) 
  • Program Evaluation Coordinator at Arizona State University (Phoenix, AZ, USA)  
  • Research Associate at MPR Associates, Inc. (Berkeley, CA, USA)
  • Senior Research Associate at Harder+Company Community Research (Los Angeles, CA, USA) 
  • Project Evaluator at Kauffman & Associates, Inc. (Silver Spring, MD, USA) 
  • Research Associate at Branch Associates, Inc. (Philadelphia, PA, USA)
  • Deputy Director of Research and Evaluation at Public/Private Ventures (Philadelphia, PA, USA)
  • Science Assessment Specialist at University of California, Berkeley (Berkeley, CA, USA)
  • Program Evaluation Specialist at Cincinnati Children's Hospital (Cincinnati, OH, USA)
Descriptions for each of these positions, and many others, are available in AEA's Online Career Center. According to Google analytics, the Career Center received more than 12,500 unique page views in the past month. It is an outstanding resource for posting your resume or position, or for finding your next employer, contractor or employee. 
 
Job hunting? You can also sign up to receive notifications of new position postings via email or RSS feed.
 
Go to the AEA Online Career Center
Get Involved
Get the most from your membership by taking advantage of the many things that you can do right now to participate in the life of the association, share your input, and promote your business.    
About Us
The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of evaluation in all its forms.
 
The American Evaluation Association's mission is to:
  • Improve evaluation practices and methods
  • Increase evaluation use
  • Promote evaluation as a profession and
  • Support the contribution of evaluation to the generation of theory and knowledge about effective human action.
phone: 1-508-748-3326 or 1-888-232-2275