I learned about the IDC storage paradox on Zoli Erdos' blog. Zoli mentions this Associated Press article, which cites IDC's estimate that "the world had 185 exabytes of storage available last year and will have 601 exabytes in 2010. But the amount of stuff generated is expected to jump from 161 exabytes last year to 988 exabytes in 2010".
Even more alarmingly, Dan Farber over at ZDNet reports that according to IBM, "the world's information base will be doubling in size every 11 hours" by 2010. Does this mean that on Jan 1, 2011, our 988 exabytes of data will double to 1,976 exabytes by 11am, and 3,952 exabytes by 10pm?
Fortunately, we don't need permanent storage for all the data we generate. For instance, spam accounted for just 8% of all emails in 2001 (said CNet); its volume rose to 36% by 2002 and 66% by 2004 (MSNBC), and is expected to exceed 90% by the end of this year (IT News). That's a huge amount of data that isn't being saved.
Still, Rich D'Ambrise from Maxell says he expects significant growth in data archiving requirements: in 2007, we will back up 75% more data than we did in 2006. But unlike IDC analyst John Gantz, he's not concerned that we'll run out of space. The storage industry is not standing still. Maxell, for instance, is beta testing 300 GB holographic disks that are no bigger than a DVD, but offer 63x more capacity. 800 GB second generation disks should be on the market by next year, and a 1.6 TB version is planned for 2010. And let's not forget stacked volumetric optical discs (SVOD); each 92-micrometer layer stores up to 9.4 GB. Available storage capacity will absolutely keep up with demand; no question about that!
The real issue is, will we store our zettabytes of data on- or offline? Rich is betting on removable media; he'd rather have mission critical data in his own possession than depend on any service provider. Zoli, on the other hand, says online is more efficient. By sharing/linking to files, we won't each need space for our own copies of the same content. Sun CEO Jonathan Schwartz says offline storage is greener ("when data's at rest, it consumes no electricity") - and easier to transport on a large scale. (As the New Yorker points out, if you made tiny chariots with DVD wheels and hitched them to snails, you'd get faster data transfer speeds than DSL.)
So, what's this got to do with web hosting? For one, you should probably monitor your oversold disk space closely. At the moment, I'm sure hardly any of GoDaddy's $7 hosting customers are using their entire 100 GB quota. But if you consider Rich's 75% growth projection, the number of customers that same 100 GB is allocated to may have to come down.
PS - Here's a GigaOM post on a 10 more fun storage facts.
Last Thursday night, I came across this SearchStorage.com article via the Storagezilla blog. Beth Pariseau wrote that Amazon's Simple Storage Service (S3) has had "performance and reliability issues serious enough" to prompt second thoughts among early adopters. In particular, SmugMug CEO Don MacAskill recently decided to move hot storage back in-house.
The instant I finishing reading the article, my RSS reader lit up with Don's response. He still loves Amazon, even if S3 hasn't solved the "speed of light problem". It takes at least 60-80ms for bytes of data to travel the distance between SmugMug's west coast location and Amazon's east coast data center. There's no getting around that. He moved hot storage closer to his web servers NOT to solve Amazon's performance problems, but to reduce those thousands of miles to inches. Don also tracked down the Storagezilla post and added a comment there.
Fast forward to this morning, when someone sent me a snippet from a Tier 1 Research news brief in which Dan Golding wrote about Amazon's disillusioned users. I gave Dan a hard time for basing his article on the same two customers Beth interviewed without giving her credit. Dan argued that attribution isn't customary in the analyst world. Besides, we shouldn't even be having this conversation. As a non-subscriber, I should have deleted any T1R content that came my way upon receipt.
Ironically, during his HostingCon presentation last year, T1R founder Andy Schoepfer's key message was "don't be an island". It's important for web hosting providers to connect customers to external ecosystems like eBay and Amazon, because no e-business can thrive in isolation. Given T1R's Hosting 2.0 advocacy, Dan's reaction seemed... Analyst 1.0-ish. But towards the end of our conversation, he did promise that an upgrade is on the way. As a point of reference, Burton Group, Dan's former employer, has a great blog that links to external sources. Same goes for Forrester. And at least 220 other research firms, including T1R parent company The 451 Group. Raven Zachary, who leads 451's open source practice, is even on Twitter!
Anyway, I'll get off my soapbox now and back to Amazon. I think every web hosting exec needs to read Don's blog post - along with Robert Cichon's post on customer satisfaction metrics. Robert said a hosting provider has done a good job if (a) the company gets written testimonials, (b) customers refer other customers because they're happy with service quality, and (c) customers defend the company against negative remarks. Amazon gets three points based on Don's reaction. What's your score?
Google Research's disk failure trends report reminds me of Ronnie, a former customer whose 10-server population suffered FOUR hard drive failures over the course of three weeks. At least a couple of the failed drives drives were brand new. Other folks with his server config experienced no such disasters. Were Ronnie's machines located in a particularly hot section of the data center? Was his utilization much higher than anyone else's?
Believe or not, after analyzing over 100,000 hard drives (parallel or serial ATA, 5400 to 7200 RPM, 80GB to 400GB) between Dec 2005 and Aug 2006, Google found that there's no significant correlation between either temperature or activity level and failure probability. Age isn't necessarily a good predictor, either. The average failure rate among 1 year old drives was ~2%, rising to ~8% in years 2 and 3, but declining to ~6% in year 4 (see page 4 of this PDF).
Also importantly, while certain SMART parameters (especially scan errors, reallocation count, offline reallocation, probational count) correlate strongly with higher failure rates, over 56% of the failed drives had zero counts on any of these variables. In other words, it's not possible to pinpoint impending failure based on SMART data.
On the other hand, drive models, manufacturers and vintages do make a huge difference. The report doesn't show breakdowns, but Amazon's Vogels says you pretty much get what you pay for. In addition to investing in high quality disks, you can also improve average longevity of your hard drive population with a longer burn-in period, during which bad disks will be weeded out.
Still, hard drive survival is very much a game of numbers, which isn't reassuring news for anyone who's running mission critical apps on standalone servers. Vogels says this is a good reason to store data on S3 and let Amazon worry about the problem; many users agree. If you're in the hosting business and would like to hold on to your customers, a virtualized storage platform is increasingly a must-have. "Fast hardware replacement" offers just aren't as compelling as freedom from hassles associated with hard drive failure.
I read about it on CNet. What you see above is Lucasfilm's data center. Its network features 300 10-gig ports and 1,500 1-gig ports. Every digital artist within the company has a desktop with gigabit connectivity.
Lucasfilm uses Verari servers with dual core, dual Opteron processors and 16 GB RAM as well as bunches of legacy machines. IT director Kevin Clark says it can take as little as 6-7 months before new equipment turns legacy. For storage, Kevin has 300 TB on a NetApp NAS. (BTW, check out this Infoworld article about NetApp's Data Ontap GX virtualization software, which allowed Lucasfilm to maintain 200 TB across 20 servers while having storage capacity appear as one single 200 TB disk. Lucasfilm has been testing the software for the past year, and will be moving to a GX cluster soon.)
As with Google, Lucasfilms says power utilization is an important factor in its data center equipment selection. The company's website also points out that its facility is LEED certified by the US Green Building Council. I've been hearing a lot about LEED these days. It's mentioned in this SearchDataCenters article, and it will be on the agenda at Data Center World. If green IT is a priority for corporate data center managers, it will likely become an increasingly important topic in your sales discussions with enterprise customers.
Jon Price from ISPCON is worried about running out of space on his Salesforce.com account. According to Salesforce's edition comparison data sheet (PDF), customers with "professional" and "enterprise" subscriptions only get 1 GB of total storage, or 20 MB per user, whichever is greater. Even "unlimited" accounts include just 120 MB per seat.
But Jon wants his CRM system to be a rich repository of data on customer interactions, with "hundreds of proposals, ppt slides, scanned documents, copies of contracts and whatnot all jammed into that same database and assigned to activities, opportunities and customer records." 120MB certainly won't accommodate this level of usage. I'll bet people will want to save screencasts and product videos as well before long.
The obvious answer, Jon says, is Amazon's S3. Why couldn't Salesforce use Amazon's web services API to build an applet for saving customer data to S3? Better yet, why not offer their own pay-per-GB storage using the servers they maintain at Rackspace?
(I didn't know that Salesforce has servers at Rackspace, but this sounds like yet another reason why Rackspace should develop an on-demand storage platform?)
Adam Gross, Salesforce's SVP of Developer Marketing, told Jon that Salesforce/S3 integration IS possible, but it doesn't seem like they have any plans to build a solution. I wouldn't be surprised to see a third party mashup in the near future. I totally agree with Jon about the tremendous PR buzz it would generate: on-demand apps + on-demand storage, what could be better? (Probably not leasing storage capacity by the server and installing/managing one's own software?)
Everyone's talking about Microsoft's soon-to-be-released Windows Home Server ("WHS"). It's a new operating system that's based on Windows Server 2003 (future editions will be based on the upcoming Longhorn Server). Microsoft's current plans are to offer it on an OEM-only basis. HP and others will start offering hardware packages later this year.
Bill Gates unveiled the WHS during his keynote at the Consumer Electronics Show. He envisions a server in every living room, so that Microsoft can "give you connected experiences 24-hours a day". It will monitor security patches/virus definitions/drive health/etc on all Windows PCs on your home network, and back up all of their data. It also offers Zune and XBox connectivity, so you can use it to store all of your music/photos/videos for easy access from any Microsoft-powered device. Wired calls it total convergence.
And of course, content stored on the WHS will be available over the Internet! The product preview mentions a "free customizable Windows Live website", and CNet reports that customers will get "personalized Internet addresses with no monthly fees". In other words, why sign up for a hosting account - to which you have to go through the trouble of uploading your data - when you can selectively web-enable whatever home-hosted content you wish?
The WHS' storage methodology is pretty nifty, BTW. Ars Technica reports that users won't see a C:\ drive - just a single, expandable storage pool. The capacity of any internal, external or USB drives on any computer can be added to the central store. WHS duplicates all data across two or more disks for redundancy. In addition to automated, incremental file backups, WHS takes scheduled snapshots of client systems, with which customers can perform full restores.
As frustrated as customers must have been by two days of much higher than usual latency and unpredictable errors, their reactions were surprisingly mild. And once Amazon pinpointed the problem (a batch of defective new hardware), the complaints stopped. In addition, three posters thanked Amazon for being transparent about sharing specific details.
In contrast, I've also had the misfortune of putting customers on new hardware that turned out to be faulty, and I most certainly didn't have the benefit of such patience and understanding. Why? The answer is in this video:
If you were making a 5 minute presentation on your company, how would you spend that time? You'd show lots of data center photos. You'd talk about 24/7 tech support. You might fill a slide or two with customer logos - but you wouldn't focus on what cool apps your users have built. Because chances are, you have no clue.
Jeff Barr's speech shows that Amazon is different. Have you ever heard of TV Mojo, he asks? They rock. What about SmugMug? They're so awesome that Jeff himself stores thousands of photos on their site. He succeeds in portraying Amazon as not just an ordinary vendor, but a collaborator in one exciting adventure after another. People sign up for S3 and EC2 not just for hosting, but to be where the action is. As Motorola CEO Ed Zander puts it, what customers want to buy are "cool experiences".
So don't let any technical problems Amazon might have convince you that you offer better web hosting. You're not on a two-dimensional playing field where "no glitches" is all it takes to be "better". Amazon will learn from any technical issues it comes across; it will get better. You may already have solved those challenges; you've been in the hosting market for much longer. But do your services have half the sizzle of theirs? And what are you doing about that??
I've been reading "Why Not?", a book about innovation. One of its key concepts is to ask yourself what Croesus (the ancient rich king) would do. In other words, if you could throw unlimited amounts of money at a problem, what solution would you pick?
Web-hosting-wise (unless you're Google), maybe a more practical approach is to ask what Robert Marsh would do. As founder/Head Surfer of EV1, Robert single-handedly created the discount dedicated servers market. He also popularized complex hosting among EV1 customers by introducing private racks, and rolled out VPSes with a BIG party, complete with fireworks.
He blogged (sort of) before blogging was fashionable (by making a personal soapbox out of his customer forum). He saw potential in APC's Mobile Data Center before Sun made a splash with the Blackbox. He wanted to travel regularly to cities with high customer density - much like what Amazon Web Services evangelists are doing. And he put some work into developing a beyond-the-box hosting environment, which so many hosting companies began offering last year.
I've compiled a quick collection of facts and stats that I think will define web hosting in 2007. (If you're reading in RSS and the slides don't show, click here.) If Robert were still in the hosting business, what would he do in today's market?
Would he build his own S3? At least two Rackspace customers have traded their managed storage for Amazon's pay-per-GB solution - and just this morning my new friend Santosh asked which hosting companies offer S3-like shared storage. What should I tell him??
Might he take Tier 1 Research analyst Dan Golding's advice and snap up a start-up CDN? Buy a shipyard and offer container colo for Sun Blackbox owners?
He'd have another Birthday Bash, that's for sure. Or several parties in different cities - he'd drive his mobile data center right up to the entrance of not just HostingCon, but Salesforce.com's and VMWare's user conferences. Did you know that 7000+ people attend each? And they work for companies that could become your customers!
The most interesting figure (on slide #18), BTW, comes from Vlad Miloushev of 3tera. Vlad thinks up to 90% of web servers are hosted in-house. Robert wanted to go after this market with not just private racks, but private suites.
What else? Who knows. If I were you, I'd take the guy to Fleming's the next time you're in Houston. He might have some insights for you!
TechCrunch reports that Smartsheet.com, a web-based project management service, is about to release new Amazon S3-powered storage capabilities:
"Previously, the company's server was located in Dallas, Texas but with the new capability to share files and send attachments, Smartsheet needed horsepower from Amazon. S3 gives developers access to storage to run their own global network, which Smartsheet needed for its December release. The new version allows users to attach documents from a hard drive or server, and send emails updates about any changes made to a document."
Where in Dallas, you ask? I did a traceroute, which terminated at Rackspace. (Just last week, I read that Webmail.us, another Rackspace customer, is also using S3 for storage.) According to Smartsheet developer Todd Fasulo:
"To clarify, we have integrated the S3 service for doc storage to our existing SAS-70 certified environment in Dallas, TX. The S3 service is used to store documents attached to Smartsheets. (It's really fast...)"
Bill Boebel from Webmail.us says he loves Rackspace. I'm sure Todd from Smartsheet does too. I was once a Rackspace customer, and their service is totally outstanding. But is Fanatical Support enough? Bill insists that his use of S3 demonstrates Amazon's success but absolutely not Rackspace's failure. On the other hand, he acknowleges in the same blog post that "yes, our use of Amazon S3 displaced our use of Rackspace's managed backups".
Yes, Bill signed up with Amazon mainly for its web services stack, an offering that's not within Rackspace's core competency. In which case, should Rackspace partner with Amazon and get a cut on these deals?
PS - Another TechCrunch reader commented that "I wish someone would start web hosting based on Amazon S3. The same pricing with CPanel, FTP, etc." That actually isn't possible. S3 offers only storage with no processing power. But apparently utility pricing does have appeal. So might it make sense for Mosso to offer pay-per-use pricing on "the system"?
Is it a success for Amazon or a failure of Rackspace? Or both? Will Amazon's offering mature to the point where it would make sense to run everything there? Will Rackspace wise up and begin to offer comparable services? Will we begin to do Grid Peering relationships where say our users or Rackspace's users could have network access from our servers to Amazon's without incurring a bandwidth charge?
Bill was unhappy with his previous backup solution NOT because it's hosted at Rackspace. Off-the-shelf backup systems just aren't very efficient for maildir, where file names change frequently (to track read/replied/flagged status), causing the same email message to be backed up multiple times. Bill wanted to write a homegrown backup system. He did so at Amazon rather than Rackspace because:
At S3, we were able to just develop the maildir backup logic and some data cleanup logic. We skipped developing the backup storage system altogether. We coded the storage client, not the storage server. Initially we had planned on building both. But when S3 came out our thoughts quickly shifted...
Bill concluded that:
We're always looking for ways to build new stuff faster. In some cases this will mean building on top of services hosted by other companies, such as Amazon. In other cases it will mean building on top of open source software and hosted it on servers at Rackspace.
So if you share Doug Erwin's ambition of winning and keeping 100% of each of your customers' hosting business, it's going to take more than the fastest hardware and the bestest service. You'd also have to match Amazon's Web Services Stack, as illustrated by Read/Write Web:
Isabel Wang. a 10 year Web hosting veteran, examines the impact emerging Web
technologies will have on the Web hosting business, and on the motivations of
hosting consumers. Click here to read her full biography.
