A photo from NASA shows Hurricane Irene hitting New York
(WEB HOST INDUSTRY REVIEW) – When choosing a reliable data center to host data in, most companies will thoroughly research the facility to determine its capabilities, including access to power grids, the types of generators and UPS used, and the level of connectivity.
But as Phillip Koblence, vice president of operations at New York Internet (www.nyi.net) explains, “the natural disaster is essentially what you are planning for when you’re building a facility.”
In the last week of August, while the rest of the eastern seaboard was frantically stocking up on food, data center operators were making their own preparations for the potential onslaught of Hurricane Irene.
Causing $10 billion in damages, the hurricane brought Category Two conditions along North and South Carolina, Category One conditions throughout Maryland and Delaware and tropical storm conditions through New York City and the Northeast.
Fortunately, the majority of data centers in the storm’s path, including New York Internet’s Bridgewater, New Jersey and New York City facilities remained unaffected by its destructive forces, thanks to mission critical infrastructures and thorough planning.
“When you run these types of facilities, you’re testing the emergency infrastructure on a fairly consistent basis,” says Koblence. “But when you catch wind that something like a Hurricane Irene is coming in, you want to do an extra test the day before just to make sure everything is running smoothly.”
When Koblence and his staff discovered that Irene was headed their way on August 23, they immediately scheduled a generator test on top of the facility’s regularly scheduled testing. The team confirmed that both the generator and UPS were running perfectly.
Another precaution the NYI team took was to ensure that the fuel tanks at both its New York City and Bridgewater facilities were “topped off.”
“What happens in these types of situations isn’t really that the automatic transfer switches don’t work; sometimes it’s the fact that you’re not really sure how much time you’re going to be without commercial power,” says Koblence. “And even if you have refuelling contracts, which obviously if you run these types of facilities you have in place, the issue is depending on how the storm affects the area you might not be in a position to accept the delivery, just from the logistics of it. The fuel trucks might not be able to pass certain roads and it might extend the amount of time that you need to rely on your initial fuel resources.”
But perhaps the most important aspect that operators need to concern themselves with in their preparations is ensuring they have the appropriate staffing, and “communicate with [their] staff that not only are they on-site, but that they have appropriate food supplies and that they’re going to be comfortable over the course of the night.”
Koblence says all NYI employees were made aware of the fact that they could potentially have to sleep in the facilities if the roads were impassable.
In New York, the subways stopped running at noon on August 27, which would prevent many employees from being able to commute to the Wall Street area data center.
When the storm finally passed, NYI’s Bridgewater facility remained relatively untouched, with the exception of some fluctuations in the voltage, which is to be expected when there are power outages.
“Those [fluctuations] were handled perfectly well by our equipment and we did not need to go on generator,” said Koblence. “We didn’t lose any service at all and we didn’t need to run on the emergency infrastructure and we were ready the entire time. We had our staff constantly checking voltage, which is the perfect precursor to any potential issue going forward. We were in constant communication with our New York facility with Con Edison and building ownership because of its presence in lower Manhattan and the potential consequences of Con Edison needing to shut the grid down in the lower Manhattan area as the result of potential flood damage.”
Above all else, Koblence said that data center operators ought to assume the “worst case scenario” where they should hope for the best but prepare themselves for the worst, particularly when considering power outages.
Since it is likely that operators will not be able to call up their generator mechanic or fuelling supplier in the midst of a storm, they need to ensure that these technicians are available onsite and that their own employees are well-trained in the mission critical infrastructure.
“I think technology is one thing, but often times, people are the hardest thing to plan for,” said Koblence. “Putting your employees first is the best investment you can possibly make in one of these situations.”
No related posts.
