November 6, 2007 -- (WEB HOST INDUSTRY REVIEW) -- Managed hosting provider NaviSite (navisite.com) has been rushing to fix an extended outage that has left thousands of its customers' websites offline for more than three days, but has slowly been bringing them back online since late Monday night.
According to a report by The Register, the interruption started Saturday as part of a planned move to replace a group of aging servers operated by Alabanza, which NaviSite recently acquired. NaviSite warned customers that their websites would be down for several hours during the migration, but unexpected issues caused the process to take much longer than originally anticipated.
Although the company has been communicating with its customers through constant updates on its website (navisite.com/sublevel.aspx?id=2017), many have expressed their displeasure with the extended outages through blogs.
NaviSite says its unable to gauge exactly how many websites were affected, but it says that Alabanza serves about 300 customers and those resell services to another 5,000 to 7,000 customers. However, some customers believe that the estimated number of affected websites is closer to 200,000, with one reseller claiming that he alone hosts close to 20,000 customers.
NaviSite has set up two numbers for customers who still don't have access to call.
COMMENTS
This has affected all Alabanza hosted sites, which is approximately 150,000 sites, netcraft has better data on this. And all services are not back online yet. Some were back online yesterday and then all was dark again this morning. They are starting to come back online at this time.
posted by: A tired host | November 06, 2007 12:59PM
Sapotek better rethink its decision to select NaviSite. This company is totally incompetent. It basically doesn't know how to bring servers up and lies about everything to boot.
posted by: Cynthia Brumfield | November 06, 2007 01:13PM
Lies and incompetence. Next thing from Navisite will be a spin trying to blame this on their customers. They wilfully pulled the plug on thousands upon thousands of business, non-profit, and I have even heard of hospitals and government sites that got yanked. No mail, no trace they had ever been online. Who knows how much economic damage their incompetence caused. We all know how their cavalier attitude and obfuscation about the restore process frustrated people trying to get an idea of when they could go back to work. Can't wait for the lawsuits.
posted by: Michael H. Bartlett | November 06, 2007 01:31PM
I am one of many business who's web site and email are still down because of navisite's incompetence. They basically decided to load all the servers in a truck, drive them from Baltimore to Andover MA and try to bring them back on line there, with out backing them up or creating a contingency plan. Once they got to the Andover site, apparently no one there knew how to hook them up. Millions of dollars are being lost by many different companies because of navisite. There should be a class action lawsuit or some legal action taken to compensate those companies for the loss of revenue.
posted by: Joseph Sherman | November 06, 2007 01:33PM
Navisite's lack of leadership and accountability is what failed this group of business owners. A utter disgrace to be truthful.
posted by: Serious Black | November 06, 2007 01:39PM
Some servers were back online as early as 5 a.m. Monday morning. Being among these, at that time we had 85% of our sites fully functional, the others affected by the DNS and routing issues. As of 9 am Tuesday, we have 100% functionality. However, we know of cohorts still at 0% functionality at this time.
posted by: Giving them the benefit of the doubt | November 06, 2007 01:41PM
Navisite is trying to downplay the number of sites involved. However, when they were bragging about acquiring Alabanza and Jupiter Hosting they proudly proclaimed 165,000 sites at Alabanza. Here's a link to their press release- should look at it before they realize it's up and they pull it down:
http://www.jupiterhosting.com/news/article/a/12/t/navisite_acquires_alabanza_and_jupiter_hosting.html
posted by: Michael H. Bartlett | November 06, 2007 01:42PM
5,000 to 7,000 customers is probably right as a number of 'client' profiles - many have multiple websites under them. The number of websites is definitely over 150,000.
posted by: | November 06, 2007 01:51PM
at 2:31 my site is still down. when will the compensation be paid?
posted by: nyaaa | November 06, 2007 02:31PM
We are a small company and we have 1400 websites. Please Navisite finishes with the lies .
posted by: Mauro | November 06, 2007 02:49PM
I contacted a reporter at the WHIR a week before the move. He told me that Navisite was a sponsor and therefore it was a conflict of interest to report on the challenges that were evident long before the move. He could have scooped everyone and given the industry a heads up but chose not to. Almost as disappointing as the fumbled migration.
posted by: Anon | November 06, 2007 02:53PM
A few notes regarding the comment by "Anon" from the editor of this site:
1) Nobody called theWHIR to offer any information about this before it happened.
2) NaviSite is not a sponsor of theWHIR, nor has it been for quite some time.
3) Even if those two things were true, we would certainly have covered an important story, regardless of whether it might reflect poorly on one of our advertisers. All of our advertisers know this, and I think many of our readers know it too.
posted by: Liam Eagle | November 06, 2007 03:05PM
NaviSite is now approaching 100+ hours of down time, many clients sites are still down ad of 4:43PM On Tuesday. I would like to call for the immediate dismissal of the NaviSite management team who allowed this to happen. We have lost well over 150 clients due to this irrisponsable decision. More Info here: http://www.webhostingtalk.com/showthread.php?t=646844
posted by: BestWeb | November 06, 2007 04:47PM
yeah, it's not so easy leaving them either, because a migration away from them at this point means tremendous issues with account setup, email configurations, not to mention database and scripting connections and hard-coded path information. Essentially, it's the ultimate nightmare of the web host and the programmer. It will certainly go down as a huge blunder, and if these guys had any sense at all, they'd simply BUY the NOC in Baltimore and keep all the techs on the job -- but it may already be too late for that... let's just pray, shall we?
posted by: Anthony Fogleman | November 06, 2007 04:53PM
I haven't gone to my day job since Friday as I have to be home to take calls from the 20 or so clients (only) for whom I have designed sites that are active. Just businesses with 2-25 people, without email, without orders, without any answers. The question is plaguing me of whether to bail now and rebuild, or sit it out one more day. Can we just say ARRRGGGHHH...
posted by: davematron | November 06, 2007 07:24PM
They seem to be down again (8:30 to 9:30 EST so far). I'm listening to the equivalent of a static-laced radio station playing Latin music as I wait for an American tech support operator to tell me what the hell is going on. The other phone lines are lighting up even as I write this with calls from irate customers. "Can't you guys (my company) keep my web site up for a few days in a row??" I understand their feelings. This is stupid. No emails from Navisite all day either. No updates, no apologies, no offers of recompense. Nothing. Just this new outage. I'm not sure what to do, but I'm holding for now... It's no surprise that their stock price has dropped 10% in the last five days. No surprise at all. The people who know about this are selling short, the rest are just like Alabanza clients.
posted by: PositronicDave | November 07, 2007 09:33PM
This has been devastating for my clients. Most are small business owners running Fishing and Dive Charter Reservation sites and a few technology storefronts.
The boats stopped going out this week - no online reservations, and I'm here with egg on my face pulling the full weight of these incompetent boobs.
My phones are ringing 24/7
Money stopped cold.
Data is in limbo.
It's quite embarrassing to me and my company. I'm not sure I can pull out of this.
It's Thursday morning, 5:30 AM, and I find it hard to believe 98% of my (only 50) sites are still down... ONLY about 50, but I built them all and feel ultimately responsible for their recovery; my hands are tied.
Good luck to everyone, I think we'll need it.
posted by: TJReynolds | November 08, 2007 05:49AM
The project Manager for the NaviSite/Alabanza datacenter migration is Jay Keating (jay.keating@navisite.com), You can thank him for this total mess!
posted by: Jay Keating is your man! | November 08, 2007 12:28PM
I helped run Alabanza for a few years. Except for a few hiccups it ran well for 8-10 years with the resellers making money and having everything automated. Why is it that companies buy well run companies and then have to assert their "we can do it better" attitude and mess everthing up. Navisite should start throw cash to the reseller so they can throw it to the website owners who have lost money...
How much money is lost in a deal like this? 30 million? 50 million? who knows .... Lesson: Bigger companies doesn't = smarter companies.
posted by: eric rice | November 09, 2007 03:14PM
Seven days later, one third of our servers were still down. We've lost half of our clients and may go out of business. This was by far the most inept business move of the year.
posted by: I may go out of business because of this | November 11, 2007 07:51AM
[POST COMMENT]
