The Risk Communicator: January 2010

by David Weinberg

During the first four years of DHS' life, Congress mandated that funds be provided to states, municipalities and selected infrastructure systems through a variety of federal grant programs. Congress also dictated that the grants be risk-based (though the first two years' grant appropriations did not use that specific terminology). It then became DHS' responsibility to "figure it out." The details of DHS models are not important for this discussion, but the need and the pitfalls are, and shed some light on when and why quantification becomes necessary.

Risk has its roots in the history of statistics. For a fascinating journey through that history, Peter Bernstein's Against the Gods is an excellent must-read. But as good as it is, it offers little guidance to those who would presume to turn the malevolent acts of man into a bunch of numbers and equations. Indeed, why do we even try?

During my sojourn into this relatively new, but ever-so-frustrating field, I was asked this question by just about everyone from the most junior staff at DHS up through the secretary, as well as congressional members and staff, mayors, governors and all kinds of risk managers. I make no claim to having a comprehensive response that everyone can (or did) buy into, but here is my case.

When you quantify something, you first try to measure it directly. For instance, wealth may be assessed by the number of dollars in your wallet. However, the direct approach isn't always possible, forcing you to find a perfect (or as perfect as possible) surrogate that can be measured, and through one or more processes derive the value of what it is you wanted to quantify. Having done that, the next step is developing a conceptual idea of how the data interact in dynamic systems -- what we call modeling.

But again: why would you want to do this anyway? What is the value added?

First, quantitative modeling forces you to think through the entire process. Mathematics has rules and when you do something mathematically you are saying that the interaction between the variables, which have very specific properties, must follow the logic represented by the equation. If your logic is wrong, the equation you write is also wrong and vice versa. If the equation you create falls apart, your logic in creating it is faulty and/or incomplete.

Second, when you attempt to quantify something, you begin to identify things about which you are uncertain. These sources of uncertainty provide clues as to the validity or falsity of assumptions you make when you develop your model. This is especially true because quantitative modeling provides repeatable results and provides a platform for playing "What if?" If the "What if" yields nonsense, there is probably a flaw in the logic; if it seems logical, you can press ahead. In this sense, quantitative models can be very dynamic, and provide unexpected insights, although you may not always like the answers you get. They also provide internal consistency checks, which while not always foolproof, can monitor the integrity of the model's logic.

But even if a consensus is reached on the importance of quantification, an additional challenge is deciding how to do it and how much is enough. Decision support models have been around for decades, as have game theory and agent-based models. But a problem common to all of these is the quality and quantity of data required by whatever mathematical technique one chooses to use. While there were many suggestions over the years by learned academics and members of Congress alike, the simple fact remained that without a formal quantification scheme, discussions or arguments on who was more or less at risk became just so many words without any meaningful way to compare. Further, the ever-present question asked by many Americans and their representatives of how much safer are we now that we've spent so much cannot and has not been answered by "just words." It still remains, unfortunately, an open question of how one can reduce risk, or what the relationship might be between dollars and risk reduction.

Still, even as the homeland security community continues to struggle with methodology, the value added to DHS and the nation so far has not necessarily been the "answers" that were computed, so much as the identification of what is important, what is needed to do a better job, and the need for rigorous methods of looking at the problem set from beginning to end. They provided a useful tool for the department to estimate the relative levels of terrorism risk between cities, states and selected infrastructure systems -- without having to play referee to disagreeing governors and mayors -- and maybe more importantly what was needed to do a better job next time. Like many things in life, the value is not in the destination so much as in the journey.

David Weinberg is President and CEO of Practical Risk, LLC. He previously served as Senior Technical Adviser in DHS's Infrastructure Protection Directorate and has done extensive work in terrorism risk analysis. He currently sits on SARMA's board of directors and serves as the organization's treasurer.