P E R S P E C T I V E S:
Evaluation & Research News
In Brief


Thank you to everyone who checked in with us after the horrible events affecting Boston and Cambridge last week; we are all okay. Our thoughts and prayers go out to all those directly affected.



Two GRG staff members are co-authors of an article on GRG's evaluation of the Schwartz Center Connections program that will be published in the July issue of the Journal of Health Care Risk Management. The Connections program is designed to build communication between healthcare generalists and specialists to mitigate adverse patient outcomes and malpractice risk.

Read more about our evaluation here.   



GRG's report on our evaluation of the Renew Boston program, for the Mass Energy Alliance, was released to the public. Our findings and recommendations are being used both internally by our client, the Mass Energy Consumers Alliance, and externally by interested stakeholders such as the City of Boston, the Commonwealth's Energy Efficiency Advisory Council, and the utilities.  




Tina Lagerstedt



Want to learn more? Check us out online!


Spring 2013    
Welcome from the President.....


I just returned from a wonderful 10-day trip traveling around Costa Rica with a group of students, faculty chaperones, and some other moms of students from my daughter's high school. We traversed that remarkably bio-diverse Central American country, marveling at the beautiful examples of natural features (e.g., tropical rainforest, waterfalls, cloud forest, Arenal Volcano, Caribbean Sea, Pacific Ocean), flora (e.g., orchids, bromeliads), fauna (e.g., toucans, parrots, frogs, crocodiles, various types of monkeys), and lush fruit crops (e.g., pineapples, bananas, mangos).


I returned to work on Monday, April 22, just in time for Earth Day, which is now celebrated annually in 150 countries around the globe. At its inception back in 1970, Earth Day was observed only in the U.S., but for the first time groups who had previously tackled only their own concerns (e.g., pollution, toxic dumps, pesticides, traffic, encroachment on wilderness, freeways) were brought together for massive coast-to-coast rallies in support of their common concern for a healthy and sustainable environment.


The phrase Pura Vida (which literally means Pure Life), is used in Costa Rican culture to signify "this is living," "excellent," "life is wonderful," or "things are going great," as well as being used for greetings of hello, goodbye, or thank you. The expression reflects their approach to life, enjoyment of their country's natural splendor, and their eco-tourism hospitality. Yet, there - as in other countries across the globe - climate change is wreaking havoc. Our tour guide mentioned that scientific models predict extreme weather changes that will affect water supply, sea level rise, volcanic activity, and extinction of some species of animals, just to name a few outcomes.


This year's Earth Day theme is The Face of Climate Change. It's a topic area in which GRG has become quite involved in recent years, with evaluations of a number of climate change education projects. I had just traveled thousands of miles but was still close to the concept of how connected are all parts of our fragile planet.

   'Irene' signature
Irene F. Goodman, Ed.D.
Founder and President

Using Time Sampling within a Learning Walks Framework
By Pamela Stazesky, Ph.D. 

Momentary time sampling has been used as a data collection tool for many years, allowing the observer to collect precise data through repeated quick snapshots. There are many benefits to using a well-developed time sampling protocol. It forces observers to focus on capturing what they see and hear, rather than interpreting what they see and hear. Thus, observer accuracy is increased, leading to high inter-rater reliability when multiple observers are used. In addition, by using a tablet or smartphone, these data can be gathered efficiently and, more importantly, without the distraction of tracking the time interval; plus further data entry is eliminated, thus reducing the likelihood of data entry errors.


Classroom observations are a much needed method to accurately evaluate implementation of K-12 educational programs. Yet they are cost prohibitive for many evaluations because they are time consuming and because of the high cost of training teams of evaluators to reach acceptable levels of inter-rater reliability. Therefore, we have borrowed from the instructional leadership arena to approach observations through a different lens. Instead of conducting full-length classroom observations, we're using the learning walk structure as another data collection tool. Learning walks are brief, nonjudgmental, focused visits to classrooms. Because they are frequently followed by feedback and reflection, they are used by school leaders as a tool to create a reflective culture and improve instructional practice.


We've conducted learning walks as a formal data collection method in our evaluations. Instead of observing an entire class period, we have focused on smaller chunks of time (e.g., 15 minutes) as the sampling unit. By limiting the time in each classroom, we were able to visit every classroom within a single day. By gathering snapshots from a large number of classrooms, we are already starting to identify patterns in domains such as student engagement, instructional practices, and classroom environment across entire schools.

Evaluation Spotlight

Recently Completed Evaluations  spotlight

We conducted an evaluation of the Harvard Graduate School of Education (HGSE) Noyce Scholarship, funded by NSF. The Noyce Scholars are part of the Teacher Education Program (TEP) at Harvard, an 11-month program leading to a master's degree in education and an initial Massachusetts teacher certification. Noyce Scholars receive a stipend and agree to teach science or math for at least two years in a high-need middle or high school. GRG's evaluation examined the benefits and challenges of participating in the scholarship program and assessed scholar outcomes by surveying all former and current scholars, conducting secondary data analysis, and interviewing program administrators and participants. 


Ongoing Evaluations

Twenty rural libraries in 15 states are rolling out Pushing the Limits: Making Sense of Science a four-part discussion series blending book club with science café. PTL discussions are facilitated by librarians and local science partners and feature a recommended popular book and two brief videos illustrating the topical theme (survival, connection, nature, and knowledge).


There are more libraries than McDonald's restaurants in the U.S., three-quarters of which are in small communities that may lack traditional informal science venues like museums.The events are gaining interest in the communities, garnering favorable newspaper coverage, and drawing people from far away as well as those who don't usually participate in book groups: "The programs brought people into the library - and into the community - who had never been here before." Discussions have been wide-ranging and lively; one librarian noted, "We had to throw people out at the end of the night." Patrons have enjoyed interacting with the professionals, who help make connections to local environmental issues.


PTL was created by a team of library professionals, scientists, and filmmakers from Dartmouth College, Dawson Media Group, Oregon State University, Association of Rural and Small Libraries, and Califa Library Group, with an NSF grant. Building on the pilot libraries' experiences, the program will roll out to 75 more libraries starting this summer.


Read more on our evaluation here

In Other News...
GRG Director of Research, Colleen Manning, presented at the AAAS conference in Boston on February 18. She spoke on a panel titled Science Festivals: Grand Experiments in Public Outreach about the findings from our external evaluation of the Science Festival Alliance's four founding member festivals. Read more about the presentation here and our evaluation here.


A Sample of a Client's Product

One of our clients, The HistoryMakers, has created the nation's largest African American Video Oral History History Collection. Their website includes oral histories of both well-known and lesser known African Americans in 15 categories, including ArtMakers, ScienceMakers, BusinessMakers, MediaMakers, MilitaryMakers, and Political Makers, all with the intent to share with the public a more inclusive record of American history.


Thank you for reading our newsletter! For more information about our exciting work, check out our website,

Find us on Facebook
View our profile on LinkedIn