from Angela Sinickas, ABC
Focus Groups - Surveys Training - Evaluation
Sinickas Communications, Inc. Tel: +1.714.277.4130 FAX: +1.714.242.7049 www.sinicom.com
This issue of Measurement Works includes practical tips on several aspects of using surveys: reporting survey numbers, analyzing survey write-in comments and communicating actions taken due to survey results. We hope you consider contributing your own suggestions and stories to share with your peers in future issues. And if you like what you see, please feel free to .
In this issue:
Client projects: Fixing survey reports that don't yet tell a meaningful story
- Tip of the month: How to use survey results when explaining business changes
My story: Susan Walker, on "waking up the left side of the brain"
- 15-minute video: Opening segment of "Extreme Makeover" workshop on how to move from being a communication order-taker to a business strategist
Article: How to analyze write-in comments from surveys
Workshops: Connecticut, New York, Illinois, Mississippi, Norway and Brazil; webinars on ROI and identifying measurement skills needed at different times
- Online forums: Benchmarking IC departments, metrics for marketing, top-rated intranets and how to measure intranets.
Discounts: $400 off ALI conference in Chicago May 20
Options for analyzing survey write-in comments
By Angela Sinickas, ABC
(Originally published Jan/Feb 2013 in Melcrum's SCM.)
Communicators have many options for analyzing survey write-in comments, each with advantages and disadvantages. The traditional way is to have a person read and categorize them by nature of the content and whether the comment is positive or negative. Figure 1 shows an example of human categorization conducted for one of my clients. At a deeper level, this report showed that within the "characteristics of communication" category, 10% commented positively on the highly informative content. "Timeliness" was mentioned by 27%, about three times as many. This report also broke out comment content by business units, locations and job levels.
Figure 2 shows an analysis of just the positive comments from the same client's survey using GoAmbition's Human Language Technology (HLT) platform. One possible output uses type size to show how frequently various key words appear in the text. This is popular with clients because it looks similar to cloud content visualization.
In this example, the human and HLT analysis of the relative proportion of positive comments on timeliness and being informative match up well, with "timely" looking about three times bigger than "informative."
"Easy" also appears relatively large in Figure 2. The human analysis showed that "easy" applied five times more often to the ease of understanding the writing than the ease of intranet navigation -- a distinction that's valuable for communicators to know. After seeing the results of the human analysis, we could adjust the software to distinguish between these two types of "easy." The software can also be used for analyzing speech, such as recorded interviews and focus groups.
Another type of software tool is exemplified by Nvivo. It shows exact quotes when searching through comments that include one or multiple key phrases. (Read the full article, including an illustration of Nvivo.)
Presenting survey results in a way that tells a story
Our profiled communicator in this issue, Susan Walker, talks about how she sees stories when she looks at survey results. So do I, but it's hard for me to find the story by just looking at the raw numbers in a data report. Once I transfer the numbers into an appropriate bar or pie chart, then the story magically materializes for me. Take a look at the two charts below that both present the same survey results. Which one is more likely to help you tell a story to your management?
To help you find the hidden story behind survey numbers, try some of these techniques:
- Use percentages, not the raw numbers of respondents.
- Summarize the net favorable and net unfavorable responses instead of showing every response percentage for every option.
- Present results in a meaningful sequence, not necessarily the order in which questions were asked.
- Use color to indicate the "good" and "bad" results. While red works well for the "bad," remember that 10% of people are red-green color blind. Try using blue for the "good" percentages.
- Use graphics, not just words with tables of numbers.
- If your response options add up to 100%, stack them into a single bar rather than presenting them side by side.
Waking the left side of a communicator's brain
Susan Walker, ABC, began her working life in the internal communication field before joining the research firm Market & Opinion Research International (MORI) to head the engagement and communication practice. She now works independently, and her book "Employee Engagement and Communication Research - measurement, strategy, action" has just been published.
It was really unfair: a detention for insolence at age 14. The new maths teacher explained that I had scored 0 out of 20 for the maths test. Impossible; it had to be impertinence. In vain did I and my classmates object that this was not done on purpose.
So, from an early age, the left side of my brain seemed to be sleeping -- figures were not "my thing." My aptitude for English led me to a career in journalism/communication. Then as communication manager for a large travel company, I was also made responsible for a new staff survey. Fortunately, I was able to draw on the experience and support of the external research agency we were working with, MORI, to help me organize the first project.
When I received my first results, I was fascinated. When I was studying a page of figures on the computer tables, a colleague passed. "How boring," he observed. "Not at all," was my response, "this is the story of our company." Indeed, that was how I saw the results; the seemingly dull figures were a gold mine of information, but they did need to be excavated for insights and understanding to recognize what was great, as well as issues to address to bring improvements. During the follow-up phase, I learned a great deal. In my innocence, I imagined that passing their results to individual managers would result in them taking action. Big mistake. So next time around I spent much time in communicating not just the results, but also how to understand them and take action.
Another discovery was the positive action that could stem from those figures. One resort manager had particularly low scores. Having been recently promoted, he was clearly finding it difficult to manage staff who had been his peers. Accordingly, the company provided advice and training for his new role; his much higher results in the next survey showed that this had worked well.
After six years in this role, I was ready to move on. But where? An organization with an open attitude to communicating with staff similar to my present job? Where was the challenge? Or a company with poor communications? That might be frustrating after the freedom I had enjoyed.
Then, out of the blue, came a phone call from the founder of MORI, Sir Robert Worcester, asking me to join his research firm. I was amazed, saying I was no researcher. However, he convinced me by explaining that they had all the research expertise; what he wanted was somebody who had developed and implemented an effective action program.
Thus I joined a research company and -- after much absorbing of the research techniques to awaken that left brain -- headed the employee research practice. My time there was both challenging and exciting; I loved combining both my communications and research experience to tell the organizational story. Indeed I still regarded myself as a communicator -- the research was just another channel.
Having left a few years ago to work independently, I also had time to write my book, and for the statistics chapter consulted one of the UK's top statisticians. Here the story comes full circle, for that statistician was one of my classmates all those years ago when I was given that detention....
aspires to be a useful resource for communicators with a need to measure, but without a great deal of time, money or expertise. Please send in
any questions you have about research and measurement, and or contribute your own experiences in having conducted measurably successful communications for others to learn from.
Angela Sinickas, ABC
Sinickas Communications, Inc.
I'll be in Europe the first two weeks of June 2013 and Brazil the first week of July with open dates on my calendar. Email me to inquire about availability for in-house training for your organization.
Catch up with
previous issues of
Go to archive
Also see issues focused on specific topics:
Global Research Tips
Tip of the Month
Connecting Survey Results
to Business Changes
As communicators, there is a lot
we can do to connect the dots for people to help them see the
outcomes from surveys they have taken during the many months between surveys. These tips can also increase the response rate for future surveys:
- Incorporate related statistics from previous employee or customer surveys whenever researching and writing about a new program, product or initiative. This reinforces that the organization is listening and making changes accordingly.
- Create forums for discussion, whether online or in-person, for your audience to brainstorm ways to improve any survey results that are lower than you'd like them to be.
- Remind your audience of changes made from previous surveys when
you invite them to take the
next survey to show them
that it is worth their time to take it.
View the opening 15 minutes of our half-day or full-day workshop "Extreme Makeover: From Communication Order-Taker to Business Strategist"
Seminars & Workshops
In the next few months, Angela Sinickas will be conducting training
on strategic planning, ROI, electronic channel measurement, and becoming
a strategic partner instead of
(See details & full calendar)
March 19, webinar on communication's ROI (SPRF)
April 6, Hollywood, FL, infusing strategy into communication (AAMC)
April 30, webinar, what you need to know about measurement (PRSA)
May 10, Stamford, CT, half-day on measuring communication and ROI (IABC/Westfair)
May 20, Chicago, measuring electronic communication (ALI)
June 18-19, Oslo, internal communications course (NCA)
June 27-29, Sao Paulo, two-day course on measurement (Syracuse University
- Sept. 30 or Oct. 1, Biloxi, MS, communication measurement (SPRF)
Advanced Learning Institute
will offer a $400 discount when you mention Angela Sinickas' name on
your registration form for ALI Conferences in 2013 where she
is a speaker:
May 20 in Chicago
Useful measurement discussions at LinkedIn
benchmark the size and
structure of internal communication departments.
A member of the Corporate Communication
forum is offering a
free guide from Vocus on 10
metrics for 2013 marketing.
(No endorsement implied; I haven't looked at it yet myself.)
A participant in the Employee Communications and Engagement
forum is looking for answers to her question about how to measure success of the portal
whether employees got what they came for.
On a related note, case studies of winning entries from the Worldwide Intranet Challenge
are available for review and discussion at Melcrum's Communicators' Network
... from advice on conducting
focus groups, to measuring
communication flow, to survey
IABC Communication World
Now in its 3rd edition,
this practical guide to
is filled with samples
of questions and reports,
as well as dozens of stories
about how measurements
have been used by
Professional Development Tools
For those who want to build their knowledge of research and measurement this year, consider some of our tools:
- Over 30, one-hour-long training CDs that include both audio and supporting documents
- Starter Kits of different survey templates and focus group guides
- In-house measurement training or our inter-active workshop called Extreme Make-over about how to have leadership see you as a strategic business partner instead of an order-taker
Buy one, get
a second one
A proven training tool, these one-hour-long audio programs are ideal for you and your colleagues to listen, read and learn. Get top communication training when you want it, on the topics you want and at the depth you want.
Buy any one of four program titles in the "Getting a Seat at the Table" series and get another title in the same series for FREE.
Mention Offer Code MM0811 on the order form. Offer Expires 12/31/13
From Angela Sinickas, ABC