Fortune Cookie Wisdom: 

"No Matter What Your Past Has Been, You Have A Spotless Future."

 Curt Johnson, CPEA, STC Senior Program Director, Richmond, TX

At least once a year, the Auditing Roundtable or a LinkedIn discussion group engages in a debate about how companies measure their audits and whether to rate findings. After all, without a meaningful means for measuring performance, companies can't tell if conditions are improving, staying the same, or deteriorating. Companies often tie performance evaluations and incentives to improvements, knowing that the employees are more likely to make the desired changes if they see some tangible benefit to them.
 

The question then, is not whether to measure for improvements, but how to effectively promote improvement.
 

To effectively engage employees and promote improvements, three things must occur: 

• The measure must relate to the performance to be improved;
• The performance must be within the control of the persons being evaluated; and
• The incentive must be meaningful to those persons.

Ineffective Performance Measurements

 
In a classic fail, many organizations separate the measurement from the performance they want improved when they connect incentives (such as bonus pay) for sales representatives to the sales agreement and not actual collection of payment for the product or services. More sales may present a rosy picture, but the company doesn't actually benefit until the transaction is completed and the fee is collected. The sales rep may even have latitude to offer discounts to increase sales. The result of course is a happy sales rep but a lower profit margin for the company.

In some situations, the improvement expectation is placed with the wrong person. Companies with drivers subject to DOT hours-of-service requirements sometimes task the trucking safety manager with keeping hours-of-service violations in check. The safety manager may have no direct control over the drivers however. It is the facility manager and the dispatcher who coordinate routes, pickups, and deliveries and tell the drivers when and where to go. The dispatcher and facility manager are often measured on the site's profits and vehicle utilization-goals that can be at odds with the drivers' time limits: "I know it's been a long day, but we need to take care of this customer now." The safety manager actually has little control in reducing hours-of-service violations.

 
Finally, lackluster incentives inspire lackluster results. Consider campaigns revolved around trinkets and logo-wear. Even when employees initially like the items, they can reach a saturation point. How many coffee mugs and T-shirts do you really want?

 
Problems with Rating Audit Findings

When I'm working with companies to develop and improve their auditing programs, whether to rate findings and audits inevitably gets discussed. I agree that it is important to help the auditee understand the relative risk and importance of different findings. A finding that presents a high risk of injury or death must be dealt with quickly (e.g., by stopping the activity or imposing appropriate controls). Similarly, facility managers need to know if any activities jeopardize their operating permits. Finally, knowing how many serious nonconformances occur per facility, before and after making changes, allows organizations to assess their improvement.

 
Management frequently disagrees, however, on whether the auditee should be measured on the number and/or severity of an audit's findings. This discussion can often take some time, so in my case there's a strong likelihood that the subject will get aired during lunch or dinner at a Chinese restaurant. It was after one such meal that I found support for my philosophy of what should be measured:

 
In my experience, audit programs that measure the performance of managers based on the number and severity of findings inevitably turn contentious. To improve their ratings, managers may conceal nonconformances from internal auditors, allowing problems to continue unabated. Closing conferences and reviews of draft reports turn into marathons as facility personnel nitpick their way to a lower-ranked finding or packaging multiple related issues into a single finding. 

At one facility, I went into the closing conference with excellent news--the facility performance was outstanding. According to this particular company's rating system, however, the top rank was reserved for outstanding performance and innovative programs. The facility had implemented the same program as another facility I had audited earlier that year. Because that methodology had been conveyed to all similar facilities as a best practice, the corporate office now expected it. The program was no longer considered innovative, and this second facility only received the second highest ranking. Instead of celebrating their outstanding performance, site management grumbled about their lower score.

 
Measuring a manager based on the number of audit findings ignores factors outside of management's control. The manager of a facility with a very detailed permit in a state or country with a complex regulatory structure may have more findings than the facility manager of a site that only requires a simple permission to operate. The number of findings may not have any relationship to which manager runs a better EHS program. 

Furthermore, the total number of findings and the presence of serious nonconformances may not reflect current facility management. When a manager and site are not performing well, EHS issues are often one aspect of the site's underperformance. Indeed, corporate and regional management may replace an underperforming manager for non-EHS reasons. On numerous occasions, I've audited a facility soon after a change in management. The new managers were making changes and implementing controls, but hadn't resolved all the issues. The new managers were there because of the high number of problems, not the other way around.

 
A New Approach

At a large waste and recycling services company I worked for, senior management knew that unresolved findings presented risk to the organization and did not present the company as the responsible corporate citizen it wanted to be. Facility managers too often demonstrated lackluster efforts to deal with audit findings when faced with other operational concerns and their perceived importance. I convinced the CEO to start assessing managers' performance using a measure called on-time completion performance, which dramatically changed the manner in which the audit program was viewed and improved the facilities' record in addressing all audit findings.

 
Our audit program, like most others, required facility management to develop an action plan. Every finding of nonconformance had to have at least one corrective and/or preventive action, with a target completion date for each action. Like other audit programs, we collected data on when actions were completed, but we also computed and reported the percentage of action items completed on time. Regardless of how many findings were identified or how many discrete actions needed to be tracked to resolve the findings, every manager could achieve a 100% on-time completion performance if they established a realistic action plan and completed the tasks when they said they would. On-time completion performance measures how well the manager can manage. 

Quarterly, audit and action plan performance was reported to facility and senior management. Rather than lengthy reports describing actions planned and actions completed (which made managers' eyes glaze over), the audit department provided one-page summaries of audits showing completion performance, as follows: 

• Total number of findings and actions;
• Number and percentage of actions completed;
• Number and percentage of actions completed on or before their target date; and
• Short description of any actions that were overdue.

Even without in-depth understanding of EHS requirements, senior management could quickly review these reports and identify which action plans were being managed well and which facilities needed attention and assistance. Within a year of reporting on-time completion performance, the measure almost doubled (from below 40% to near 80%).

Holding only the facility manager accountable to resolve action plan can sometimes result in conflicting priorities. Supervisors and senior managers directly influence facility and local managers by assigning tasks, setting objectives, and approving budgets and compensation, so senior managers need to be responsible for completing EHS audit action plans as well. After all, reducing risk through resolution of all audit findings is important not just to facility managers but the whole company.

 
To include supervisors and senior managers in the resolution process, we produced reports rolling up on-time completion performance for all the audit action plans for facilities and activities under each senior manager. That manager received copies of individual reports for each site for which he or she was responsible. That manager also received a rating compiled from the on-time performance for all the action items for all of their facilities. The more activities and properties under a senior manager's direction, the greater the number of audit action plan items compiled into the rating.

 
Seeing the improvement in on-time completion performance and recognizing the company's desire to resolve further audit findings, we linked the on-time completion performance measure to bonus compensation for both facility and senior managers to promote further improvements. Other operational measures such as sales realized or volumes managed were used to determine the potential bonus for a manager. The percentage of actions completed on time was then multiplied by the potential bonus, meaning that their actual bonus could range from 0 (if nothing was completed on time) to 100% of their potential bonus (if everything was completed on time). Understandably, facility managers worked hard to complete all their actions on time so they could receive their full bonus, and senior managers provided full support. On-time completion performance of EHS audit action plans approached 100% after being linked to compensation.

By measuring facility managers on their ability to complete an action plan, rather than the specific number and nature of the findings themselves, the audit program became focused on resolution and improvement. Appropriately, pressure remained for auditors to correctly distinguish findings of nonconformance from opinions and improvement opportunities, but auditors and facility personnel disagreed less often about whether identified issues should be included in the audit report. Further, managers wanted the auditors' opinion of the severity of the finding. Their job was to assure ongoing performance and improvement of the company's assets (e.g., people, permits, property). They needed to know the relative risk of their audit findings to determine how best and how quickly to resolve them. The focus on resolution and improvement inspired more managers to collaborate on solutions.
 

Previously it seemed that every site was on its own. With senior management compensation impacted by on-time completion, however, they were much more likely to lend support staff, form teams, and coordinate budgets to design and implement solutions that could be applied across the organization, reducing overall costs.

The audit team became accepted as part of the company's overall improvement efforts. Managers started asking to have their sites audited, recognizing that the audit process helped them present their best face to regulators and the community-good publicity before expansion permit hearings, for example. Rather than keeping the audit program in the shadows, senior management started touting the company's audit program as an important element of the internal verification and improvement system that regulators and communities could rely on.

Conclusion

So why was use of the on-time completion performance measure so successful? The three elements to effectively promote performance improvement worked together: 

• The measure directly related to the performance of something the company wanted to improve: resolution of action plans;
• The performance was within the control of the persons being evaluated: the facility managers and their supervisors controlled how and when the action plans were completed; and
• The incentive was meaningful to those persons: compensation.

In addition to assuring resolution of all audit findings, the company wanted to reduce the number and severity of findings at its facilities. Over time, the audit program was able to show a decrease in the number and severity of findings. This was a result of managers being given the tools and understanding to develop action plans that both corrected problems and prevented their recurrence. With incentives to implement the action plans on time, managers collaborated on solutions and exchanged best practices, resulting in improvements across the company.

 
When the incentives are meaningful and the activity being measured is within their control, people will try to accomplish what is expected. That's why assessing managers based on their ability to effectively manage, as measured by their on-time completion performance, will improve your organization's ability to timely and effectively resolve audit findings, even if this measure is not linked to compensation. Measuring on-time completion performance will achieve the fundamental goal of auditing: to improve your organization.