A six-day exercise last month marked the third disaster recovery test conducted since August.  Each test has resulted in important lessons leading to stronger restoration capabilities.

The most recent test was the largest yet in both duration and scope.  Beginning on February 8 and concluding on February 13, the test lasted 120 hours, exceeding the August (96 hours) and October (48 hours) exercises.  Ten agencies participated, compared to nine in August and four in October.  In addition to the highest agency participation, the test also had the largest number of participating service providers: IBM, AT&T Retail, AT&T Wholesale, and Dell all took part.

The test included 51 applications, topping totals from August and October (41 and nine, respectively).  Last month's exercise was also the first to test T0 and T1 applications together.  T0 applications are those that are to be recovered within 32 hours; T1 applications are to be recovered within one week.  Six applications that previously failed were successfully restored.

Approximately 90 percent of all servers were restored, as opposed to 70 percent last August.  In addition, many physical servers were restored to virtual hosts during the test, requiring fewer physical servers for the exercise and allowing for more comprehensive testing.

Another first - IBM's Tivoli Service Manager (TSM) tool was used for the first time to recover applications for the departments of Juvenile Justice and Driver Services, whose servers are located at the state data center.  TSM automates data backup and restore functions.

The test identified issues with servers that are clustered to support a single application due to size or complexity.  Because restoration is more complicated in a clustered environment, a test environment or "sandbox" will be established at the state data center in preparation for subsequent testing.

The next test, a 150-hour exercise, is scheduled for August 2011.

Back to top