Measurement Works

  from Angela Sinickas, ABC

 

February, 2013

Strategic Planning  

Focus Groups - Surveys Training - Evaluation  

 

 Sinickas Communications, Inc.   Tel: +1.714.277.4130   FAX: +1.714.242.7049                                        www.sinicom.com               

Greetings!

This issue of Measurement Works includes practical tips on several aspects of using surveys: reporting survey numbers, analyzing survey write-in comments and communicating actions taken due to survey results. We hope you consider contributing your own suggestions and stories to share with your peers in future issues. And if you like what you see, please feel free to . 

 

In this issue:

  • Client projects: Fixing survey reports that don't yet tell a meaningful story
  • Tip of the month: How to use survey results when explaining business changes
  • My story:  Susan Walker, on "waking up the left side of the brain"
  • 15-minute videoOpening segment of "Extreme Makeover" workshop on how to move from being a communication order-taker to a business strategist
  • Article: How to analyze write-in comments from surveys
  • Workshops: Connecticut, New York, Illinois, Mississippi, Norway and Brazil; webinars on ROI and identifying measurement skills needed at different times
  • Online forums: Benchmarking IC departments, metrics for marketing, top-rated intranets and how to measure intranets.
  • Discounts: $400 off ALI conference in Chicago May 20

 Article 

 Options for analyzing survey write-in comments

A Sinickas
 
By Angela Sinickas, ABC  


(Originally published Jan/Feb 2013 in Melcrum's SCM.)

 

Communicators have many options for analyzing survey write-in comments, each with advantages and disadvantages. The traditional way is to have a person read and categorize them by nature of the content and whether the comment is positive or negative.  Figure 1 shows an example of human categorization conducted for one of my clients. At a deeper level, this report showed that within the "characteristics of communication" category, 10% commented positively on the highly informative content. "Timeliness" was mentioned by 27%, about three times as many. This report also broke out comment content by business units, locations and job levels.
  Human analysis

 

Figure 2 shows an analysis of just the positive comments from the same client's survey using GoAmbition's Human Language Technology (HLT) platform. One possible output uses type size to show how frequently various key words appear in the text. This is popular with clients because it looks similar to cloud content visualization. 
 
In this example, the human and HLT analysis of the relative proportion of positive comments on timeliness and being informative match up well, with "timely" looking about three times bigger than "informative." 
Cloud visualization

"Easy" also appears relatively large in Figure 2. The human analysis showed that "easy" applied five times more often to the ease of understanding the writing than the ease of intranet navigation -- a distinction that's valuable for communicators to know. After seeing the results of the human analysis, we could adjust the software to distinguish between these two types of "easy." The software can also be used for analyzing speech, such as recorded interviews and focus groups.

                                             

Another type of software tool is exemplified by Nvivo. It shows exact quotes when searching through comments that include one or multiple key phrases. (Read the full article, including an illustration of Nvivo.)   

  Client Project:  

Presenting survey results in a way that tells a story

  
Our profiled communicator in this issue, Susan Walker, talks about how she sees stories when she looks at survey results. So do I, but it's hard for me to find the story by just looking at the raw numbers in a data report. Once I transfer the numbers into an appropriate bar or pie chart, then the story magically materializes for me. Take a look at the two charts below that both present the same survey results. Which one is more likely to help you tell a story to your management?
 
To help you find the hidden story behind survey numbers, try some of these techniques:
  • Use percentages, not the raw numbers of respondents.
  • Summarize the net favorable and net unfavorable responses instead of showing every response percentage for every option.
  • Present results in a meaningful sequence, not necessarily the order in which questions were asked.
  • Use color to indicate the "good" and "bad" results. While red works well for the "bad," remember that 10% of people are red-green color blind. Try using blue for the "good" percentages.
  • Use graphics, not just words with tables of numbers.
  • If your response options add up to 100%, stack them into a single bar rather than presenting them side by side.
  
Hard to interpret data presentation  More meaningful data presentation

 My Story

Waking the left side of a communicator's brain

Susan Walker  
Susan Walker, ABC, began her working life in the internal communication field before joining the research firm Market & Opinion Research International (MORI) to head the engagement and communication practice. She now works independently, and her book "Employee Engagement and Communication Research - measurement, strategy, action" has just been published.

 

It was really unfair: a detention for insolence at age 14. The new maths teacher explained that I had scored 0 out of 20 for the maths test. Impossible; it had to be impertinence. In vain did I and my classmates object that this was not done on purpose.

 

So, from an early age, the left side of my brain seemed to be sleeping -- figures were not "my thing." My aptitude for English led me to a career in journalism/communication. Then as communication manager for a large travel company, I was also made responsible for a new staff survey. Fortunately, I was able to draw on the experience and support of the external research agency we were working with, MORI, to help me organize the first project.

 

When I received my first results, I was fascinated. When I was studying a page of figures on the computer tables, a colleague passed. "How boring," he observed. "Not at all," was my response, "this is the story of our company." Indeed, that was how I saw the results; the seemingly dull figures were a gold mine of information, but they did need to be excavated for insights and understanding to recognize what was great, as well as issues to address to bring improvements. During the follow-up phase, I learned a great deal. In my innocence, I imagined that passing their results to individual managers would result in them taking action. Big mistake. So next time around I spent much time in communicating not just the results, but also how to understand them and take action.

 

Another discovery was the positive action that could stem from those figures. One resort manager had particularly low scores. Having been recently promoted, he was clearly finding it difficult to manage staff who had been his peers. Accordingly, the company provided advice and training for his new role; his much higher results in the next survey showed that this had worked well.

 

After six years in this role, I was ready to move on. But where? An organization with an open attitude to communicating with staff similar to my present job? Where was the challenge? Or a company with poor communications? That might be frustrating after the freedom I had enjoyed.

 

Then, out of the blue, came a phone call from the founder of MORI, Sir Robert Worcester, asking me to join his research firm. I was amazed, saying I was no researcher. However, he convinced me by explaining that they had all the research expertise; what he wanted was somebody who had developed and implemented an effective action program.

 

Thus I joined a research company and -- after much absorbing of the research techniques to awaken that left brain -- headed the employee research practice.  My time there was both challenging and exciting; I loved combining both my communications and research experience to tell the organizational story. Indeed I still regarded myself as a communicator -- the research was just another channel.

 

Having left a few years ago to work independently, I also had time to write my book, and for the statistics chapter consulted one of the UK's top statisticians. Here the story comes full circle, for that statistician was one of my classmates all those years ago when I was given that detention....   
Measurement Works aspires to be a useful resource for communicators with a need to measure, but without a great deal of time, money or expertise. Please send in any questions you have about research and measurement, and or contribute your own experiences in having conducted measurably successful communications for others to learn from.

 

Sincerely,

Angela Sinickas, ABC

Sinickas Communications, Inc.


I'll be in Europe the first two weeks of June 2013 and Brazil the first week of July with open dates on my calendar. Email me to inquire about availability for in-house training for your organization.

Join Our Mailing List
Archive
Catch up with

previous issues of
Measurement Works:

 Go to archive

 

Also see issues focused on specific topics: 

 

Social Media  

 

Calculating ROI  

 

Global Research Tips  

 


Tip of the Month

  Meshing gears   
Connecting Survey Results
to Business Changes
 
  As communicators, there is a lot
we can do to connect the dots for people to help them see the
outcomes from surveys they have taken during the many months between surveys. These tips can also increase the response rate for future surveys:
  • Incorporate related statistics from previous employee or customer surveys whenever researching and writing about a new program, product or initiative. This reinforces that the organization is listening and making changes accordingly.
  • Create forums for discussion, whether online or in-person, for your audience to brainstorm ways to improve any survey results that are lower than you'd like them to be.
  • Remind your audience of changes made from previous surveys when
    you invite them to take the
    next survey to show them
    that it is worth their time to take it.  
Read the entire article 
with more tips.

 

15-Minute Mini-Workshop
Extreme makeover

View the opening 15 minutes of our half-day or full-day workshop "Extreme Makeover: From Communication Order-Taker to Business Strategist"

Seminars & Workshops

In the next few months, Angela Sinickas will be conducting training
on strategic planning, ROI, electronic channel measurement, and becoming
a strategic partner instead of
an order-taker.

(See details & full calendar)

 

  • March 19, webinar on communication's ROI (SPRF)
  • April 6, Hollywood, FL, infusing strategy into communication (AAMC)
  • April 30, webinar, what you need to know about measurement (PRSA)
  • May 10, Stamford, CT, half-day on measuring communication and ROI (IABC/Westfair)
  • May 20, Chicago, measuring electronic communication (ALI)
  • June 18-19, Oslo, internal communications course (NCA) 
  • June 27-29, Sao Paulo, two-day course on measurement (Syracuse University
  • Sept. 30 or Oct. 1, Biloxi, MS, communication measurement (SPRF)  

$400 OFF  

 Advanced Learning Institute 
will offer a $400 discount when you mention Angela Sinickas' name on
your registration form for ALI Conferences in 2013 where she
is a speaker:

 

May 20 in Chicago

 
Online Discussion Forums

Useful measurement discussions at LinkedIn

benchmark the size and
structure of internal communication departments
.
 
A member of the Corporate Communication forum is offering a
free guide from Vocus on 10
metrics for 2013 marketing.

(No endorsement implied; I haven't looked at it yet myself.)
 
A participant in the Employee Communications and Engagement forum is looking for answers to her question about how to measure success of the portal and
whether employees got what they came for.
 
On a related note, case studies of winning entries from the Worldwide Intranet Challenge
are available for review and discussion at
Melcrum's Communicators' Network.

Measurement Manual

Measurement Manual
"A definitive
hands-on resource
... from advice on conducting
focus groups, to measuring
communication flow, to survey
question construction,
it's all here."

 IABC Communication World

 

Now in its 3rd edition,

this practical guide to

communication measurement

is filled with samples

 of questions and reports,

as well as dozens of stories

about how measurements

have been used by

communicators.   

 

Author 

Reviews

Order Form

Professional Development Tools

For those who want to build their knowledge of research and measurement this year, consider some of our tools: 2013-01 Audio Kits
  • Over 30, one-hour-long training CDs that include both audio and supporting documents
  • Starter Kits of different survey templates and focus group guides
  • In-house measurement training or our inter-active workshop called Extreme Make-over about how to have leadership see you as a strategic business partner instead of an order-taker
2013-01 Extreme Makeover  

 

Buy one, get
a second one

Free! 
"Getting a Seat at the Table"

Communication Training Programs (audio and documents)   

 
A proven training tool, these one-hour-long audio programs are ideal for you and your colleagues to listen, read and learn. Get top communication training when you want it, on the topics you want and at the depth you want.

Buy any one of four program titles in the "Getting a Seat at the Table" series and get another title in the same series for FREE. 

Mention Offer Code MM0811 on the order form.       Offer Expires 12/31/13 

Header 1

Measurement Works 

From Angela Sinickas, ABC

February, 2013