This article has been updated to include an emailed statement from 123-reg.
UK hosting provider 123-reg is continuing to recover data for VPS accounts affected by connectivity issues that started on Saturday and resulted in “some data loss for some customers.”
123-reg director Richard Winslow emailed affected customers on Sunday, according to Slashdot. The email described the issues that had knocked the VPS platform offline. It does not appear to offer customers any credit for the issue.
“As a result of our team’s investigations, we can conclude that the issues faced having resulted in some data loss for some customers. Our teams have been and continue to work to restore,” Winslow said in the email, ensuring affected users that it had left “no stone unturned” in the investigation.
The issue highlights the importance of disaster recovery, and also how crucial it is to understand what kind of backup is included with a hosting package. Without reading the fine print or ensuring you have a plan B should an outage occur, you could set your business up for major loss of data, not to mention the resources needed to restore websites back to normal.
“For those customers with their own backup of their settings and data, if you wish to restore services yourself you can do this by issuing a reimage command through your 123 Reg control panel, this will give you a freshly installed VPS on a new cluster, where you can restore your service.”
A report by The Register identifies one of the affected customers as software company InnMaster. In a blog on its company site, InnMaster said it has been a 123-reg customer for two years and hadn’t experienced any other outages in that time.
“This is a major embarrassment for a large web hosting group and judging by what we have seen on Twitter it has left some of their customers who had no backup or emergency plan with no website and potentially no business any more! [sic] – We can assure you, we are not one of those,” InnMaster wrote. The company took matters into its own hand by migrating its data to a mirrored server in a new location.
“A new platform will be available by the end of the year for customers which we will provide self-managed and automated snapshot backups, in addition to architecture technology to backup the whole platform, something that is not available on the current platform. I hope this goes some way to win back your confidence.”
Customers can stay updated on data recovery efforts by checking its status page.
Update, 04/19/2016 9:30 am ET:
“We suffered a technical fault, which has affected a minority of 123 Reg customers. The fault was limited to 67 servers out of 115,000 (across Europe) and it is important to note that only a selection of customers on the 67 servers affected has been impacted. We are investigating the restoration of each VPS on a case-by-case basis and are working individually with customers to keep them informed of the website recovery process. We are working as quickly as we can to restore service to normal.
Our VPS product is an unmanaged service and we always recommend that customers implement backups to safeguard against unexpected issues. Customers who had purchased 123 Reg backups can be online now as can those who are using another solution for website backup. If customers restore from their own backups, this will not overwrite our efforts. Additionally, customers who have restored from their own backups are now hosted on new servers. By using new servers, we will ensure that we do not overwrite the previous servers and impact the data recovery process. We are working with Kroll, the leading data recovery specialist to manage the process of restoration.
We would like to extend our apologies to affected customers and assure them that we are doing everything we can to restore their data as quickly as possible. We are keeping customers informed as we restore their websites.”