|
|
Subscribe |
Sign up here to receive the TCEC newsletter direct to your inbox every quarter!
| |
|
Welcome |
It's been two years now since the Tobacco Control Evaluation Center launched the first issue of this newsletter, Process & Outcome. Happy anniversary! We hope you've enjoyed reading about how evaluation can be utilized to move your objectives forward. Our goal has been to answer your evaluation questions, stimulate new ways of looking at things and introduce areas for improved practice.
As we continue to do that in coming issues, we plan to shake things up a little by trying out some new ideas of our own with regards to layout, design and format. So you can expect to see a new look with our next issue.
We hope you'll like it!
Robin Kipke Editor |
|
|
|
|
 Getting to Yes! The Art of Persuasion
We here at the Tobacco Control Evaluation Center are always talking about how evaluation is meant to be used -- to inform your program (and other key players) about where people are in terms of awareness or support of the issue, to serve as evidence which can be conveyed to decisionmakers, and to leverage this evidence into action. However, data alone does not create social change, as many of you know firsthand. People have to be persuaded to act upon their feelings, and it is up to organizations like yours to build public will into action.
A nifty little guide called Discovering the Activation Point: Smart Strategies to Make People Act from http://www.activationpoint.org offers a number of ideas to consider as you begin thinking about how to use your outreach, data collection and messaging most effectively in order to achieve your project objectives. It could be a particularly useful resource as your organization develops its communication plans -- including use of social media. (See related news about the Social Media manual currently being written.)
Here are the main points from chapter 1 which relates to evaluation and use of data collection results:
The target is likely smaller than you think. Often you only need to motivate a few key people to achieve the change you want. Most efforts are aiming too broadly at too large of an audience.
Ask the decisionmakers how many is enough to get their attention and change their behavior. Find out what their pressure point is (through key informant intervi ews or other means). Identify who they pay attention to and are influenced by.
Focus on those you can actually persuade. Find out who is with you and who is against you and then start with those who are not actively opposing you -- people who might listen to what you have to say.
Segment audience targets until you can't segment anymore. It's critical to understand who is in your target audience and what characteristics they share and differ in. Find out what they care about so that you can choose the approach that will resonate best with them.
Target the audience(s) with the greatest influence over your decision maker. Identify the decision makers first and figure out who do they listen to most. While using youth as spokespeople can attract media interest, they hold less sway over public officials than adult constituents because they can't vote. On the other hand because of their consumer habits and buying power, they are very influential with business leaders.
Find and activate social reference groups. Target those who others look to for leadership and determination of when to take action. When you persuade these folks, others will follow.
Show strong public support by picking audiences that are willing to show (not just voice) their support. People who are willing to go public with their support create the perception that there is broader support for an issue.
Test the activation point before launching a full-scale campaign. People's actions don't always match up with their beliefs or opinions, so it's important to figure out what they will actually carry out to ensure your efforts to build momentum work. One way to maximize this is to align your issue with what they care most about and then ask small test groups to respond in some way. Then you can replicate the strategy that worked best with your larger audience with better results.
For more details about framing your advocacy messages more persuasively, you can obtain a download of the publication by registering at the site through the link above. It just may give you some fresh ideas!
Photos from I & E days
back to top
|
|  Lessons Learned -- Not Just for Your Final Evaluation Reports As many of you already know -- at least you tobacco-control veterans -- one of the most important parts of your final evaluation report is the "lessons learned" section. And, although progress reports do not specifically require it, documenting what your project learned from the intervention and evaluation activities in each reporting period is highly recommended. In fact, doing so along the way while the details are fresh in mind is the best means of making the most of all of your project's hard work and allows you to build on that experience for the road ahead. The rationale for including a "lessons learned" section is fairly straight-forward: to let project personnel reflect on what was learned in the process in an effort to make things easier or more doable in the future. Additionally, this valuable information can be transferred and conveyed to other projects that may be working on a similar objective to assist them in their efforts. This is particularly the case with the hiccups that are inherent in so many of the objectives pursued by projects. Knowing what obstacles may come up and strategies to handle them (or to avoid) can save your colleagues from wasted effort. In previous newsletter pieces we have recommended documenting organizational history in terms of the processes by which a local project takes on an objective from beginning to end. This is vital for a host of reasons. For one, the lifecycle for plans is fairly uniform. While the actors, or contacts, may change with each objective, the processes used to achieve the goal is often fairly similar -- conduct outreach, establish relationships with key players, gather informative or persuasive data, present the data, try to get decisionmakers to act. By documenting the process of how your project went about this -- who you worked with, what specific tactics were tried, the sequence and timing of actions, what other factors played a role -- the information can serve as a rough game plan that can be followed in future plans. Another reason to document lessons and the process of pursuing objectives is staff turnover. In analyzing the last two cycles of final evaluation reports, we have found that staff turnover has been a major challenge to many projects. It is natural for people to move on and find new positions and new careers. When they do, they can often leave a major void in the organization and the forward motion of the campaign if there is no documentation of the process along the way. It is important for your project to know what efforts have already been made, what tactics have been tried, and which contacts and relationships are in place so that you don't have to start over from scratch when someone leaves. 
Documenting the lessons learned during a "crisis" situation can also be vital. Although it is especially difficult to record the process during a time when your project encounters a major roadblock or even a train wreck, it is critical to identify what went wrong, the factors that led to it and what you could have done differently so that later plan cycles can learn from these experiences. Having a key member of the team regularly provide a blog or mini-report that describes the situation can be very worthwhile. We have so often seen that one cycle's "failure" to reach an objective often results in successes in later cycles because of the valuable information that is often gleaned when things do not go so well. The key is learning from past mistakes. O ne strategy for documenting organizational strategies and lessons learned is to create a shared data file where team members can briefly describe the path they took and make recommendations for the future. Consistency over time is key. If all team members were required to spend 20 minutes a week jotting down this information, at the end of a three year cycle there would be wealth of information which could inform not only your final evaluation report, but also future plans. This data can be generated by individuals or in staff meetings where team members can share information on how obstacles were overcome, and what they would have done differently if they were to do it again, and so on. And of course, this documentation doesn't need to be high tech. It could be as simple as keeping handwritten notes in a binder or in a Word document kept on a shared drive. Whichever the method used, it should be something where team members and the project director can easily access it. Learning from past successes as well as mistakes makes for a much quicker road forward. So take a few minutes right now to document your project tactics and experiences. It will most likely save your project a lot of time and effort in the future! Photos by: istockphoto back to top |
|
Training Delivered the Goods on the Principles of Survey Design
This spring, two of TCEC's evaluation associates, Robin and
Travis, embarked on a tour of California to provide a series of trainings on "Designing Effective Surveys." As projects were gearing up for their data collection (including developing their data collection instruments), the team at the Tobacco Control Evaluation Center thought a training on designing surveys would be timely and beneficial to project directors, evaluators and health educators.
The one-day trainings were held at four different locations throughout California. One webinar-based remote training was also offered to those who could not attend in person. In total, over 81 people participated in the sessions, representing over 50 local projects.
The training focused on the pragmatic approaches to designing effective surveys, with collaborative breakout sessions devoted to end-use strategizing, survey question creation, and a collective feedback session on questions devised by each of the small groups. This latter portion of the day was based on 10 of the principles espoused by survey guru Don Dillman. During this segment participants learned the value of balancing question stems and answer choices with positive and negative frames, avoiding do uble-barreled questions, alleviating respondent burden, and so much more. By the end of the day, the Dillman principles dripped from the pores of training participants!
At the conclusion of the training, participants had the opportunity to meet individually with Robin and Travis about their specific evaluation questions. Some people even brought the instruments that they were currently working on in order to get feedback.
Participants received a notebook full of useful resources, including sample surveys, guidelines on survey construction, an end-use strategizing worksheet, and information on field testing instruments. The day's presentation slides were also included as part of the "goodie" folder. These resources are now available for download on our website under Past Trainings.
Robin and Travis were incredibly impressed with not only the turnout for these trainings, but the great attitudes and good cheer among the participants. Like our previous trainings, getting out in the "field" gave us the chance to learn about the issues faced by local projects in developing evaluation instruments and collecting data, as well as to see some old friends and meet new ones.
For those of you who attended the training, we at TCEC would LOVE to hear from you and learn about how you have been able to implement the survey design strategies into your own work. So, give us a call or email us to let us know!
Photo by: istockphoto
back to top
|
|
|
 Upcoming Events & Releases
Culture in Evaluation: Public Recognition of Your Efforts A few years ago, TCEC presented a statewide workshop on culturally competent evaluation in Sacramento where, together, participants thought through the cultural challenges that present themselves during evaluation. The knowledge generated during the workshop about various priority populations served as the beginnings of a series of cultural guides to help us in our tobacco control work in California.
Thanks to this effort and the subsequent research and further input from programs, TCEC has been able to create the "Culture in Evaluation" documents that are available individually on its website and collectively in the just released TCEC publication, Evaluation in Context. The process and importance of developing these guides has been recently documented in an article appearing in the peer-reviewed journal Health Promotion Practice, titled "Developing Culturally Competent Evaluation Tools with Tobacco Control Program Practitioners." We invite you to read the article on our website. Health Promotion Practice has chosen the article to be included in an online continuing education course for health professionals, and asked TCEC to develop test questions for this purpose. In one of our future webinars, we will talk about some of the issues in the paper and allow participants to test themselves with the Health Promotion Practice assessment. So, stay tuned. We will continue thinking through culture in evaluation. After all, one of the paper's conclusions is that culture is always changing and the effort for culturally competent evaluation is ongoing.... New Manual on Using Social Media in Development As more tobacco control projects begin to consider using social media as a means to promote their efforts, questions naturally arise about which medium will best suit their purpose, how to use it effectively and how to evaluate what difference this is making. To address those questions, TCEC is partnering with the California Youth Advocacy Network (CYAN) to develop a manual that covers the wide variety of social media your organization could utilize. The manual should be ready for release early next year, so stay tuned. In the meantime, if you have any questions about social media, you can contact Amelia Silbert-Geiger at CYAN, amelia@cyanonline.org.
Share the Good Word about Our Newsletter Is there anyone in your organization that should be receiving this newsletter but is not yet a subscriber? The Tobacco Control Evaluation newsletter, Process & Outcome, was designed to be useful to project directors, evaluators, staff members and coalition participants -- in fact, anyone involved with the data collection and evaluation activities of California tobacco control projects. So if you know someone who should be receiving and reading this quarterly newsletter, please forward them this issue and encourage them to sign on as a subscriber (at the top of the page). It is entirely free and we won't share your email address with others. Don't miss out! Sign up to receive our newsletter!
Hellos and Goodbyes 
Recently our good friend and Evaluation Associate Cathy Lemp retired from her position here at the Tobacco Control Evaluation Center. Her dedicated research and interviewing skills will be greatly missed.
We are fortunate, however, to have Margarita Garcia Hernandez, a native of Spain, join our center as a temporary graduate researcher. With a background in public health and bioinformatics, Margarita is working on creating a searchable database of data collection instruments that users will be able to access through our website. We are happy to have Marga's expertise (and Spanish language skills) added to our team!
back to top
|
|
 We are the statewide technical assistance center on evaluation for all CTCP-funded projects in California Project directors, their staff, evaluators and subcontractors can call or email our evaluation associates for individualized assistance with questions about their tobacco control-related work with regard to:
- Developing evaluation plans
- Creating or adapting data collection instruments
- Analyzing data and interpreting results
- Writing up your final evaluation report
Access our collection of useful evaluation resources via the web where you can:
- Download our Tip Sheets for "how-to" information on evaluating tobacco control interventions
- Watch short training presentations on data collection methods
- Find our Culture in Evaluation guides for suggestions on working with priority populations
- Listen to archived recordings of past TCEC webinars
WE'RE HERE TO HELP! back to top
|
|
|
|