August 16, 2006 -- (WEB HOST INDUSTRY REVIEW) -- In the Web hosting business, uptime is essential. A serious outage in the data center almost invariably leads to serious outrage among customers, who are counting on the promises of 99.999 percent uptime offered by more or less every hosting company in the world to be met.
And in most cases, when faced with a major service outage and widespread customer mutiny, a Web hosting company will retreat from what is usually considered a "PR disaster." While the flood of customer questions can understandably overwhelm a Web host, many issue incomplete announcements, clouding the problem or shifting blame, often leaving customers with a vague and incomplete account of the problem while they await updates that may or may not come.
When Web hosting provider DreamHost (dreamhost.com) experienced a major meltdown during the month of July, however, the company took a very different tack when owning up to the issues in its data center. The company published a detailed account of the problems on its blog, explaining the sources of the problem and DreamHost's efforts to prevent a reoccurrence. And, perhaps not altogether surprisingly, the posting was met with a widespread, and mostly positive, response from customers.
Compounding the damage of the outage was the fact that DreamHost was still earning back whatever goodwill it may have lost when the company suffered a similar outage in September of 2005, when a confluence of ISP outages and power failures brought its data center offline.
In a post that began with "what a three weeks..." DreamHost CEO Josh Jones described a series of incidents that, throughout July, caused the company weeks of problems culminating in several outright outages at the company's data center.
"As I'm sure most of you already know, we've had nothing but troubles, large troubles, for pretty much the last three weeks," he wrote. "A lot of these troubles were our fault, a couple of them were at least ostensibly beyond our control, and they all compounded each other."
DreamHost, says Jones, operates out of a data center owned by Switch and Data in the Garland Building in Los Angeles. Before the company's original 2005 outage, he says, the facility informed tenants that power capacity for the building had been met, and with no upgrades planned, it would be difficult to continue building within facilities.
Despite the limitations of the facility, an obviously better option was not available, so DreamHost began seeking out means within the same building to guard against future outages, eventually arranging space in a data center "down the hall" from Switch and Data, operated by Alchemy.
After negotiating connections with Switch and Data, DreamHost set up equipment in the Alchemy data center, and immediately began experiencing configuration and packet-loss problems, many of them - by Jones's own admission - DreamHost's fault. After considerable re-configuring and reworking, the company solved the setup problems.
"On Saturday, July 22nd," wrote Jones, "the building lost power."
Even when power returned, it was at reduced levels and did not support the building's cooling systems, which made it difficult to keep servers online. This outage also produced serious and widely reported downtime for personal networking giant MySpace, which also occupies part of the Garland Building, and chose to keep its servers offline until cooling was restored.
"On Friday, July 28th, we lost power again," wrote Jones. "This time, we were able to get our entire system back up much quicker and with close to no problems. Of course, it had been less than a week since our last power outage."
The network and power troubles continued for DreamHost, which managed to get its services back to normal operations around the beginning of August.
Jones's unusually detailed explanation prompted an outpouring of opinion, with more than 400 replies posted to the blog entry, most of them by customers, and most of those expressing solidarity, encouragement and even gratitude.
"I'm not going anywhere, and this blog post is a big reason why," wrote a user named Kevin in a posting very typical of the general tone. "Thanks for explaining what's going on and being up front about everything."
The posting also caught the attention of marketing bloggers, who praised the company for being so forthright in a situation that would cause more than a few firms to withdraw or make excuses.
In his link to Jones's post, marketing writer Seth Godin said there were important lessons about customer service to be learned from the experiences of DreamHost.
"Lesson one:" he wrote, "when things get messed up, being clear, self-critical and apologetic is really the only way to deal with customers if you expect them to give you another chance."
•Webmasters may not be able to manage large websites with problems faced on account of sharing huge bandwidth
•Customers experience unexpected bandwidth problems encountered and are quite a costly affair.
•Costly incurred on additional charges for lending the extra bandwidth to the customers.
•Shutting the site causes customers to loose huge business posted by: Palcomweb | April 25, 2008 06:18AM