The Planet Explosion Causes Outages

  • By theWHIR.com , June 02, 2008
  •   Digg
      Delicious
    submit to reddit  Reddit
      Newsvine
      Stumbleupon
      Twitter
    Facebook

    (close)

    From:
    To:
    Share | Send | Print | Comments (28)

Verio Attracts Resellers with Free Month of VPS Hosting: Companies looking to boost revenues are finding Verio's Free VPS promotion the right fit for getting started as Hosting Reseller.

By Justin Lee, theWHIR.com

June 2, 2008 -- (WEB HOST INDUSTRY REVIEW) -- Dedicated hosting provider The Planet (theplanet.com) suffered an explosion, 4:55 pm CST on Saturday at its H1 data center in Houston, Texas, which led to an outage that affected approximately 9,000 servers and 7,500 customers.

The explosion and accompanying fire knocked down three walls surrounding the facility's electrical equipment room, as well as destroyed the underground cabling that powers the first floor of the facility. Fortunately, there were no injuries, nor were any servers damaged or lost.

The company has been in constant communication with its customers -- an essential aspect of any disaster recovery plan -- through its forum (forums.theplanet.com) and customer portal, as well as other popular forums like Web Hosting talk (webhostingtalk.com).

In an earlier post yesterday on The Planet's forums, Kevin Hazard said the "short was in a high-volume wire conduit" and that the fire department instructed the company to not "activate [its] backup generator plan."

After the fire marshall inspected the facility The Planet was allowed to restore power to the facility. The company immediately turned back on the generators to receive power on the second floor.

Then, earlier this morning, the company successfully brought back 90 percent of the second floor servers after more than 40 hours of troubleshooting. Meanwhile, support technicians are currently working on location to manually restore the remaining 10 percent on the second floor, as well as restore power to the first floor.

Despite this, many customers, namely those companies who have lost sales, customers, and other revenues, remain upset by how long the recovery process is taking and have actively voiced these criticisms on forums.

One WHT member C0bra writes: "I'm fed up waiting for more information about these terrifying issues. Communication is poor and every minute my site is down is costing me $1000s in ad revenues. This is probably going to cost me a $10,000,000 sponsorship deal with a major brand advertiser."

UPDATE: Given the contentious nature of the user C0bra's claims (as far as the comments section on this story are concerned), it is worth pointing out that the user has previously commented on Webhosting Talk that he or she currently has no servers hosted at The Planet. This would seem to suggest that this particular story of lost money quoted above is very probably untrue.

Douglas Erwin, The Planet chairman and CEO, responded to these complaints late Sunday evening onThe Planet forum.

He writes: "I would like to extend my sincere thanks for your patience during the past 28 hours. We are acutely aware that uptime is critical to your business, and you have my personal commitment that The Planet team will continue to work around the clock to restore your service.

"We understand that you will be due service credits based on our Service Level Agreement. We will proactively begin providing those following the restoration of service, which is our number priority, so please bear with us until this has been completed."

The Planet added a new advanced backup server in March, which marks the company's  largest and most powerful backup server yet.

OLDER:  Rackspace Adds New Firewall Controls | NEWER:  SoftLayer Reduces Packaging by 80%

Comment anonymously or log into your WHIR account

Logging in allows enhanced commenting features (such as external linking) in news, features, blogs and more.

User:

Pass:

(reset password)

Don't have an account yet? Register now!


 

Comment by Anonymous on Monday, June 02, 2008

Regarding that c0bra person.. if he's really raking in $10,000,000 advertisers, he should invest some of that money in round-robin DNS, redundancy in terms of multiple data center locations, etc. Idiot.

Comment by Anonymous on Monday, June 02, 2008

c0bra, yeah. Sweet. WHT is such a font of useful information. Not.

Everyone is losing "thousands of dollars an hour" every time there is a hosting outage. Doesn't matter if they are a $4.99 a year site that gets 10 visits a month. You know, that next visit could be the MILLION DOLLAR DEAL!

What a tired lie. People really need to think of something new to say during an outage..."Hey, my brain surgeon can't access my brain scan images! He's in the middle of surgery on my brain now! Get the server back up!"

That's a good one. Try that next time c0bra.

Comment by Anonymous on Monday, June 02, 2008

I think it goes without saying at this point that multi-million dollar businesses that rely on a web server to be online are exactly what disaster recovery is for. This Cobra guy's claims seem, at the least, to be an obvious exaggeration. However, I'm sure the lost-ad-revenue impact of this outage, if it could be calculated, would be astonishingly high.

Comment by Anonymous on Monday, June 02, 2008

Liam I agree with you especially since some ad exchange businesses are hosted there. I've learned a lot from this: from now on our TOS and a checkbox at signup will make it clear our clients are NOT getting redundant hosting unless they order it. Some will still blame us, but you can't please every bargain hunter.

Comment by Anonymous on Monday, June 02, 2008

I feel that c0bra needs to take a reality check and assess how he operates his business. If he is telling the truth, I think he needs to invest in a guru that will bring his site(s) up on multiple servers in seperate data centers running redundant round-robin DNS. But then again, he's probably full of crap.

Comment by Anonymous on Monday, June 02, 2008

c0bra guy is an idiot and the typical kid that runs off to WHT whenever their server dies for X reason and says the same crap. What kind of person would run their entire operation out of 1 DC. He claims to have his operations running on multiple servers and has backups blah blah blah but he keeps them all in the same DC on the same floor in the same state. If he truly is making the kind of rev's he is claiming then it would be pocket change to run out of multiple sites across the US or world. I really feel sorry for his clients as he must have a great scam going on them to allow this type of very poor design to host their ad campaigns and I would hope at least one of them will read this or his WHT posts and call him on his BS claims.

10K deal not a 10M deal should be setup this way.

Comment by Anonymous on Monday, June 02, 2008

I guess next time people go spouting off about loosing millions of dollars they will think twice, your complaint could end up on WHIR for everyone to see what a fool you are.

Comment by Anonymous on Monday, June 02, 2008

This suck bad, what the hell happened, we lost the db on www.outaouaisweb.com Hurry and fix this please ... a.s.a.p.

Comment by Anonymous on Monday, June 02, 2008

i have 2 servers hosted with them that have been down since this started saturday. one came up for about an hour or so this morning as some mail trickled in, then back down again. with loss of sales i hope the compensation offered is decent...

Comment by Anonymous on Tuesday, June 03, 2008

On s'en Calice
merci
bonsoir !

Comment by Anonymous on Tuesday, June 03, 2008

Il y a-t-il quelqu'un qui peut traduire en francais un résumé et quand ca va être arrangé?

Comment by Anonymous on Tuesday, June 03, 2008

Do we have a time-frame on when this unfortunate problem will be resolved?
Thank you......

Comment by Anonymous on Tuesday, June 03, 2008

Une partie du cablage electrique des serveurs a été detruit par une explosion... Les techniciens travaillent 24h/24h pour remettre en ligne leur serveurs. Presque tout les serveurs du 2ieme étage ont été remis en ligne et ils travaillent maintenant sur ceux du premier étage. Il demandent d'etre patient jusqu'a ce que tout soit rentré dans l'ordre... C'est a peu pres ce que ca dit.

Comment by Anonymous on Tuesday, June 03, 2008

Pis c'est quand que ça va revenir dans l'Ordre? Parce que moi je commence à être pas mal tannée!

Comment by Anonymous on Tuesday, June 03, 2008

C'est arrivé samedi a 4h55... Ca peux encore prendre un petit bout! Je peux essayer de les contacter pour avoir un update sur la situation et je l'écrirai ici...

Comment by Anonymous on Tuesday, June 03, 2008

C'est arrivé samedi a 4h55... Ca peux encore prendre un petit bout! Je peux essayer de les contacter pour avoir un update sur la situation et je l'écrirai ici...

Comment by Anonymous on Tuesday, June 03, 2008

Je viens tout juste de leur ecrire un mail... Je vous tiens au courant...

Comment by Anonymous on Tuesday, June 03, 2008

Does anyone knows the estimated time left for a 100% server recovery? Can it possibly take 3 more days?

Comment by Anonymous on Tuesday, June 03, 2008

http://forums.theplanet.com/index.php?s=999beeb8acce402a0790e9dd08fba95f&showtopic=90185&st=40

Comment by Anonymous on Tuesday, June 03, 2008

I am a small time operation running 1 quad core box and they are basicly ignoring my issues at the planet. My ticket got closed saying my box has power. That is fine and dandy, but everything is inaccessable. They are ignoring the little guys here.

Comment by Anonymous on Tuesday, June 03, 2008

It's just torture being down, then up, then down again, then up again, then down again...I have to question whether or not the place is salvagable, or they should just start hauling the servers elsewhere...

Comment by Anonymous on Tuesday, June 03, 2008

They are trying to fix the problem. The time they take to answer everyones little question is time that they don't put on the repartation. If everyone ask questions like you do, they will keep answering and not reparing.

Be patiente, you are not the only one waiting.

Comment by Anonymous on Tuesday, June 03, 2008

Sandra : To be honest with you, I don't think the technicians nor the ingenieurs are the only doing the clicks and pressing the spacebar.

Comment by Anonymous on Tuesday, June 03, 2008

"with loss of sales i hope the compensation offered is decent...posted by: primo" -- Oh, this is another good one. "I lost SALES and REVENUE! You owe me a million dollars!" Good luck with that.

Comment by Anonymous on Tuesday, June 03, 2008

Got to love this beautiful world of technology. To many people whining.

Comment by Anonymous on Tuesday, June 03, 2008

Given all the response to the quote from the person called C0bra, I thought I'd point out down here also that we added a note to the story about a previous WHT post where that user claimed to have no servers at The Planet. Seems like they might just be somebody who enjoys piling on.

Comment by Anonymous on Tuesday, June 03, 2008

I think that the planet did a great job posting updates to their forum through the day. I checked in also via phone to see what all was going on, they did a good job IMO.

We have most of our servers at the Dallas NOC but our backups run from their Houston H1 DataCenter.

I know that the "frequent updates" didn't add a whole lot of comfort to those that were affected, BUT it could have been a lot worse if they were completely shut out as to what was happening.

Bottom line, The Planet is a good company and I don't think that they could have done a whole lot else to keep everyone up to date except maybe do "twitters" which are kind of handy...

Comment by Anonymous on Thursday, June 05, 2008

Ditto to the "ignoring the little guy" comment above - I'm still down, & the tech support guy on the phone said no end in site. They are working reboot tickets in the order received. I didn't open a ticket until Tuesday - figured what's the point, they are aware of the problem, right? Kind of restating the obvious... Now I am still down - 300 sites offline - with supposedly 9000 tickets ahead of me. They are just too big and too few people to physically push all the buttons.