Peer to Peer Magazine

61 WWW.ILTANET.ORG Communication is critical in a crisis. Sidley lawyers were most interested in their contact list from Exchange/Lotus Notes. They needed access to email addresses and phone numbers to communicate with business associates and loved ones, but the New York-based Exchange and Notes servers were destroyed in the towers. Keeping test and disaster recovery systems current is critical in an emergency. Being able to restore backup tapes and proceed with recovery efforts should be viewed as a necessity, baked into every project plan going forward and audited regularly with fully tested DR and recovery drills. Backup, backup, backup. Although the days of backup tapes are waning, it is of extreme importance that engineers follow a strict protocol for ensuring that backups are executed as scheduled and restoration procedures are fluid, simple and exercised regularly. Despite solid processes, Sidley lost all data from its New York City offices on 9.11.01. The firm's data from Monday, September 10, 2001 was never recovered because the backup tapes were not resident in the New Jersey warehouse with all of the previous tapes. Use common sense. Recovering the data in Chicago made the most sense, despite potential latency connecting New York City- based users to a Chicago backend (in 2001, domestic latency issues were of concern). The IT leadership team decided to base the restoration effort in Sidley Austin's Chicago data center, despite the fact that 50 percent of the document management and email teams were stranded in New York. The firm's New York City data center in WTC was destroyed, along with desktops for Brown & Wood users residing in WTC (Sidley users' desktops were still in midtown Manhaan and were not directly affected by the disaster). Above all else, teamwork. Working together in a coordinated fashion was critical to the success of the recovery effort. Every morning LESSONS LEARNED Rising Strong: A 9/11 Disaster Recovery Case Study thereaer at 8:00 a.m., the team gathered for critical updates. Risks and challenges were exposed, and informal exchanges between interdependent teams proved invaluable in many ways. Divide and conquer. Sidley's CIO and information services directors each assumed leadership for their respective areas of responsibility, applications were prioritized and teams were formed. The CIO led the communications effort with aorneys and staff; she manned the phones, provided critical information to concerned employees and their loved ones, and assumed responsibility as the external face of the project. The director of infrastructure led the network restoration effort and defined the engineering requirements to promote the Chicago-based test environment to production in time to receive the backup tapes from the New Jersey warehouse. The director of applications led the applications team and the data restoration efforts across each dependent system. Applications were prioritized, with Exchange and iManage being mission critical to restore ASAP. Other applications were put into a bucket to be prioritized at a later date. The Baker Robbins team led the Exchange and Notes restoration effort. Because the email environment included two production systems, an emergency Notes to Exchange migration was planned and executed so that all New York City- based employees could get into Exchange, even if they were previously Notes users. Baker Robbins also led the document management team, executing the PC DOCS and iManage system restores, including all documents that formerly resided in the New York data center. Like the email team, the DMS team had to plan and execute an emergency data migration from DOCS to iManage so that Brown & Wood users could access their documents that had been edited as recently as September 9, 2001. 9.8.2001 Sidley applications team supporting Lotus Notes, Exchange, PC DOCS and iManage moved Sidley servers (running Exchange and iManage) to Brown & Wood data center (Lotus Notes and PC DOCS) in the WTC. Server move proved successful; the document and email management teams remained in New York City to support the production environment in the relocated data center at the firm's downtown location. 9.11.2001 8:45am: Word of the crisis spread from London to Chicago and elsewhere. Things kicked into motion. 4pm: Sidley's IT leadership team gathered at CIO's office in Chicago for a strategy/ planning session. The team determined that recovery should occur in Chicago and not New York City. 8pm: Leadership team disband in Chicago. Communications continued constantly. The prescription was for rest while teams in Chicago awaited the backup tapes to come from New York City to Chicago.

• Cover