A new report from the National Research Council of the National Academies (NAS) was the focus of extensive discussion at SARMA's recent Fourth Annual Conference on Security Analysis and Risk Management. The report, commissioned by Congress in 2008, examined six risk analysis models and processes and concluded that "it is not yet clear that DHS is on a trajectory for development of methods and capability that is sufficient to ensure reliable risk analysis other than for natural disasters." SARMA was fortunate enough to be able to invite to the conference Stephan Parker, a senior program officer with the Transportation Research Board of the National Academies, and Robert Kolasky, Assistant Director with DHS's Office of Risk Management and Analysis, to detail the take-aways from the report.
 
Mr. Parker began his presentation by noting that, while this particular report is new, the National Academies has been involved in government-wide discussions about risk management since before 9/11, and that after the attacks it released a number of publications citing the importance of science and the spirit of scientific inquiry in combating terrorism, especially when considering emergency response and infrastructure protection.
 
The National Academies' Review of DHS's Approach to Risk Management was a natural next step in the organization's ongoing efforts. The tasks included evaluating "the quality of the current DHS approach to estimating risk and applying those estimates in its many management, planning, and resource-allocation (including grant-making) activities" and assessing "the capability of DHS risk analysis methods to appropriately represent and analyze risks from across the Department's spectrum of activities and responsibilities, including both terrorist threats and natural disasters."
 
The process was lengthy and comprehensive, Mr. Parker explained, with dozens of offices in the private and public sector consulted. In addition to hearing from staffers at DHS and other government agencies, the National Academies heard testimony from employees of SAIC, Cox Associates and ABS Consulting.
 
Having closely examined DHS's risk management approach, the report's authors listed 10 areas of necessary improvement:

• Availability and reliability of data.

• Modeling the decision making and behaviors of intelligent adversaries.

• Appropriately characterizing and communicating uncertainty in models, data inputs and results.

• Methodological issues around implementing risk as a function of threats,vulnerabilities and consequences.

• Modeling cascading risks across infrastructures and sectors.

• Incorporating broader social consequences.

• Dealing with different perceptions and behaviors about terrorism versus natural hazards.


• Providing analyses of value to multiple, distributed decision makers.

• Varying levels of access to necessary information for analysis and decision making.


• Developing risk analysis communication strategies for various stake-holders.

Robert Kolasky, an assistant director with DHS's Office of Risk Management and Analysis, responded to the report's criticism by placing it in the context of the department's efforts to implement "integrated risk management" as charged by DHS Secretary Janet Napolitano in May. "The NAS study showed us how we collectively should get on a better trajectory to improve our risk analysis capability," Mr. Kolasky said. "The challenges NAS identified about what makes risk management and risk analysis so difficult -- we took those as proven priorities." 

To address those issues, DHS took the above list of 10 challenges identified by the National Academies report and noted areas where they intend to make "general specific progress" and use that progress to implement technical solutions. "What we're trying to do is get better in-house, bring people in to work on these technical problems, get more people into the department who are capable of doing homeland security risk analysis," Kolasky said.
 
Nevertheless, after checking off various areas of intended progress, Mr. Kolasky admitted that it was "a perhaps overly optimistic take on how much we are doing to address the challenges identified by NAS, but if forced to we could point to something within each of these checks and say we're trying to make progress," he said. "We're not there, we understand were not there, we're trying to learn and systematically make progress in that direction." 

In an effort to combat terrorism in the United States, the U.S. Coast Guard (USCG) developed the Maritime Security Risk Analysis Model (MSRAM) to analyze and manage risks from direct and exploitation attacks by terrorists. MSRAM enables relative risk comparison across all of the National Infrastructure Protection Plan (NIPP) critical infrastructure and key resource sectors. The MSRAM database contains thousands of critical infrastructure targets and scenarios (target/attack mode pairings) across the nation's ports and waterways, scored by USCG security experts with respect to Threat, Vulnerability, and Consequence. National resources including consequence modeling and security studies, intelligence data and reliability engineering techniques support the analysis. MSRAM also provides capabilities to evaluate risk mitigation strategies at the tactical, operational and strategic levels. MSRAM is institutionalized in USCG policy, informing resource allocation decisions at every level, and was recognized with the 2006 Joel Magnussen Innovation Award for Management.

The MSRAM program is currently being extended with a new framework called Maritime Security Dynamic Risk Analysis and Management Application (MSDRAMA). MSDRAMA leverages MSRAM's quantitative risk assessment data and methods in scenario-based "what-if" simulations that project the likely impacts of maritime counter-terrorism strategies over time. It also captures estimated life-cycle costs and timetables for deploying such strategies and achieving risk reduction. By combining these outputs, MSDRAMA enables USCG decision-makers to assess the cost-benefit (and time-benefit) trade-offs for alternate strategies across a range of plausible future situations and identify robust security risk management options.

Dynamic Risk Management: Key Limitations Addressed

MSRAM evaluates alternate risk reduction strategies individually at the asset level, via discrete before/after "snapshots" of risk. In contrast, MSDRAMA assesses combinations of security strategies, and projects how risk exposure is likely to be reduced continuously over time. It also enables comparative "what-if" analyses assuming that various aspects of the security "landscape" might change in the future. In effect, MSDRAMA provides a virtual environment for practicing risk management strategies and learning from simulations rather than costly investments with unknown effectiveness.

Individual security strategies generally only address a subset of attack modes and their attendant risks. For example, a single patrol boat can counter a single small boat, but cannot engage and defeat an attack
involving multiple boats or a large hijacked vessel. However, MSDRAMA supports construction and testing of portfolios of security strategies, including acquiring resources and personnel, training, and improving allocations and tactics to deploy new and existing assets.

This raises the question of how to combine risk reduction contributions from multiple independent strategies that impact a given scenario. MSDRAMA ascertains that as risk is reduced, it becomes
progressively harder to achieve further gains: more effort is required to achieve the next level of improvement. Thus, it may be prudent to avoid the assumption that risk decreases linearly as successive
security strategies are applied against risk. Accordingly, as MSDRAMA projects increasing risk reduction from original levels, it progressively discounts estimates of security strategy impacts, resulting in a nonlinear model.

Since budgets and assets are constrained, risk management decisions hinge on tradeoffs among alternate risk reduction strategies. MSDRAMA allows analysts and decision-makers to compare the simulated
values of key performance metrics to identify the relative strengths and weaknesses of different strategies. MSDRAMA currently tracks and projects three strategy key performance metrics to inform strategy
trade-off analyses for decision makers:

• How much risk does a security strategy reduce?

• How much risk does the strategy reduce per dollar of investment (expected cost efficiency, ROI)?

• How soon and at what rate does a strategy reduce exposure to risk (expected time efficiency)?

Risk management strategies are developed based on specific assumptions about risk and funding, in the short and long term. As strategies are implemented over time, these situational factors continue to evolve, often outside of government control. These changes may potentially invalidate key assumptions underlying a strategy, however reasonable they were at the initial point of decision.

In particular, adversaries observe our actions to improve domestic security and adapt, typically by changing their intended targets and tactics, or developing capabilities to overcome security measures. That is, deterrence is transitory, so strategies must be adaptive in order to defeat adaptive adversaries.

MSDRAMA addresses the first challenge by facilitating "lifecycle" decision support: as time passes, analysts periodically update scenarios based on the best available intelligence (and execution results to date). MSDRAMA then re-projects the chosen strategy into the future. If outcomes continue to be favorable, the strategy has been re-validated. If not, MSDRAMA acts as an "early warning system,"
alerting analysts promptly to emerging problems, helping them isolate variances from initial assumptions, and enabling them to define and validate suitable mid-course corrections in security strategies.

MSDRAMA addresses the second challenge of adaptive adversaries in a similar fashion, by enabling analysts to create diverse scenarios that anticipate potential terrorist responses to proposed security
strategies. For example, scenarios can incorporate assumptions as to when terrorists are likely to detect improvements in our defenses and explore how (and over what duration) they are likely to modify their
targeting tactics and attack capabilities. The resulting simulations provide a war gaming capability for testing and tuning strategies to ensure that they are robust against plausible terrorist adaptations.

Finally, risk analysis is typically performed on individual targets (e.g., vessels, port facilities, commercial installations). However, investments in risk reduction strategies often apply more broadly to a number of targets (e.g., regional communication, situational awareness, evacuation planning, etc.). MSDRAMA bridges this gap by rolling up its key performance metrics from targets and strategies to ports and Captains of the Port (COTPs).

MSDRAMA Software Solution

MSDRAMA employs a model-simulate-analyze software framework that the USCG has evaluated in a prototype project focused on risk management of the small boat threat. In this evaluation, the HQ MSRAM Team considered solutions that addressed vulnerabilities and consequences as well as USCG tactical solutions to increase effectiveness and capacity of USCG boat patrol operations in a major port. These diverse risk management solutions employed one or several improved capabilities combined across the Prevent-Protect-Respond-Recover continuum. The various security solutions analyzed addressed risk reduction by mitigating vulnerabilities, accounting for Prevent and Protect.

Additionally, the consequence reduction solution focused on Respond. Future analysis may look at the long term consequence reduction and infrastructure recoverability of various resiliency solutions, and can assess the cost and time efficiency of such solutions in a similar manner. The software projected outcomes via four key performance metrics - risk reduction, total lifecycle costs, cost efficiency (ROI) and time efficiency, allowing commanders to compare alternate strategies and make policy-level trade-offs.

The next step in this prototype is to analyze and help manage risks of transferring terrorist material and personnel. The intended MSRAM/MSDRAMA configuration will start overseas, cover international maritime transport modes, and end in US ports. MSDRAMA allows decision makers to 'test drive' solutions across a range of alternative possible futures and identify robust security strategies. MSDRAMA's underlying model/simulate/analyze paradigm and performance metrics are not inherently tied to maritime terrorism risks. This flexibility opens the door to applying these methods and software tools to risk management problems facing other DHS agencies, such as aviation, highway, and border security, as well as to security challenges facing other critical infrastructure networks, such as systemic risks to financial markets.

LT Eric Taquechel works for the Domestic Port Security Evaluation Division (CG-5412) at Coast Guard Headquarters. Mr. Jeff Fuller works for ABS Consulting in support of CG-5412. Dr. Rich Adler is Founder and Chief Architect of DecisionPath, Inc., also working in support of CG-5412.