I came across Uptime Institute founder Ken Brill's CIO Magazine article via 3tera VP Marketing Bert Armijo's blog.

Ken says while hardware prices are falling, total cost of data center ownership is headed through the roof. 5 years from now, the purchase price for a rack of servers will drop 27.5% from $138K today to just $103K. But while it only takes 15 kilowatts to power that rack right now, the energy requirement will rise to 22 - 170 kilowatts by 2012. It could cost as much as $2.3 million to power/cool $103K worth of gear throughout its 3-year lifespan.

(I'm not sure if this figure includes switches and routers and such. A recent Cisco/APC/Emerson study shows that servers/storage/cooling consume 76% of data center power, with 11% going to networking equipment, 3% lighting, and 10% power conversion losses. If Uptime's calculations didn't take the other 24% into account, Ken's $2.3M becomes over $3M!)

I've been thinking about Ken's stats and trying to understand what they mean. As a point of reference, I was looking at Dell's website, which advertises the 4U PowerEdge 6950 dual core, dual processor Opteron server for about $9K. Is Ken saying that:

(a) This particular machine will cost 27.5% less 5 years from now?

(b) 2012's late model machines will sell for 27.5% less than what's on the market today?

(c) The amount of server hardware that fills up 4U of space will be available for $6500 in 2012?

If we assume he means (c), and we accept Sun's claim that "server performance, power and space efficiencies are improving at up to 40% annually on average, and could double every 2 years", then 4U of space may be able to accommodate not one but 4 servers that each feature 4x more processing power and 4x greater energy efficiency.

In other words, $6,500 could buy you 16x more computing resources than that dual Opteron! If that's the case, you might even be able to afford $1M per rack per year in electricity. But only if you virtualize like crazy. No more leasing data center space per square foot or per rack. No more dedicated servers, either. The average customer won't need 4x more processing power in 5 years, which means you won't be able to justify turning on a whole entire server just for them.

You'd also have to replace hardware early and often. Sun recently announced a refresh service for swapping out your servers at least 3 times over 42 months. At first I thought that sounded wasteful, but if server power efficiency is improving at 40% per year, holding on to old gear might end up costing you more. Again, virtualization would be a must. You wouldn't want customer apps to become attached to machines that will be phased out before long.

Bert from 3tera says changes in data center economics will make it increasingly difficult for enterprise CIOs to justify operating their own facilities. But they won't outsource to traditional colo or dedicated server providers. Instead, he agrees with Cassatt CEO Bill Coleman that in the near-ish future, you'll be "paying for data center horsepower the same way you pay for electricity or gas". I think so too. How about you?

PS - On a somewhat related note, eWeek says Intel will release its "Clovertown" chips today. The quad core processors have a 50 watt thermal envelope, versus 80-120 watts on earlier models. That's a 38-60% drop.

PPS - Also, speaking of the Uptime Institute, check out this SearchDataCenter.com interview on how they've helped The Planet save $10K/month on electricity. The Planet, the article says, is looking to expand beyond Texas into the Midwest.

In early 2001, just a few months before Exodus filed for bankruptcy, Robert Marsh launched Rackshack. Unlike his struggling competitors, who typically built servers to spec, Robert sold $99 Cobalt RaQs. Only one configuration was available, and orders were provisioned instantly and automatically. And instead of demanding multi-year commitments, Rackshack offered month to month service. By the time I joined the company in early 2003, Rackshack (which later changed its name to EV1Servers) had become the world's largest dedicated server provider.

A year or so later, Robert unveiled EV1's private racks program during a customer gathering; two attendees signed up on the spot. Soon other orders starting pouring in, along with complicated network diagrams and super detailed server specs from customers who wanted their systems built just so. We did our best to accommodate any and all requests, which were a huge challenge to keep track of. Only much later did I learn about ITIL from Rich Bader over at EasyStreet. By that time, Amazon had already launched S3 and would soon introduce EC2.

Unlike EV1's Custom Order team, who gladly built whatever customers asked, EC2 sells only $0.10 virtual server instances. There's just one configuration available, and orders are provisioned instantly and automatically. Instead of demanding month-long commitments, Amazon offers pay-as-you-go service in 1 hour units.

According to Vinne Marchanadi from Deal Architect, pay-as-you-go is what large customers nowadays are looking for. (A former Gartner analyst, Vinnie now advises enterprise IT buyers on vendor selection.) He offers the analogy of plugging into an efficient power source versus buying fancy generators. On behalf of his clients, he says:

"Message to vendors - so long as you meet our security, privacy and compliance standards, we want as vanilla, standardized a service as possible. Sell us capacity by unit of consumption. We want to leverage all your economies - in financing, procurement, operations, everything. In return, we want to fit as much as possible in to your standards."

Another couple of years from now, will standardization again give way to customization? I think the answer is yes. And no. Amazon recently started offering Machine Image sharing. And VMWare's virtual appliance marketplace features about 400 listings. And SalesForce.com offers over 500 partner apps on AppExchange. And earlier this month Netvibes unveiled its universal widget API... It seems service delivery platforms will become more - not less - standardized, while each user will have increasing freedom to mix and match a wide range of interoperable applications into highly customized solutions. Doesn't that sound like the best of both worlds?

Robert Cringely says Google is on its way to become the biggest CDN ever. It controls more network fiber than any other company in the world - and it's building two huge data centers in South Carolina alone. Google's enormous capacity, coupled with end users' increasingly voracious appetite for all things web-based, will lead to a future in which Google will act as a giant proxy server for the Internet. It will be "our phone company, our cable company, our stereo system and our digital video recorder"...

Greg Linden disagrees. He thinks that instead of cornering the market on bandwidth, Google is trying to build a world of infinite storage and CPU power. Well, *and* bandwidth.

No web hosting provider will have access to Google's economy of scale. Or out-of-this-world local government incentives. (Unless you're a state-licensed operator in China, but that's a different story.) Sorry, there isn't and won't be any way around this. BUT one resource that Google won't be dishing out in buckets is... knowledge.

I came to this realization while reading this article on MySpace's infrastructure. Once again, I spotted it on Data Center Knowledge!

MySpace was originally built on Perl + Apache + MySQL, but before the site went live, its developers switched to Windows + Cold Fusion + MS SQL. Later on everything was rewritten in C# + ASP.NET. The site started out with just one database, then write vs read transactions were split between one master DB and two slaves. Soon this evolved into vertical partitioning, or a separate database for each and every feature. Finally its technical team settled on running separate database instances for each block of 1 million accounts. All logins come through one single front-end DB, which redirects each user to the database containing files associated with his account.

In the beginning MySpace had server-based storage, then it built a storage area network with room for more disk drives. At one point the company had two full time technicians (!) manually distributing SAN resources between database instances. Finally it moved on to a 3PARData's virtualized storage solution, on which all disk drives can be accessed as one single pool of capacity. In 2007, MySpace plans to replicate its SAN (which is currently in LA) in two other locations, to eliminate its dependency on one single data center location.

Other major changes MySpace has made to its infrastructure include a caching layer between its web and database servers. In addition to minimizing DB lookups, machines on the caching tier are used to store temporary session data, which aren't given permanent database space. Also, Myspace was among the earliest adopters of MS SQL Server 2005, the better to take advantage of its 64 bit support. The 32-bit SQL 2000 limited MySpace to 4GB RAM per server. Now its standard config is 64 GB RAM.

Throughout endless trials and errors, MySpace's developers were under enormous pressure to keep its service online. Could they have benefited from having a team of Internet infrastructure experts to bounce ideas off of? Comparing their experience with benchmarking data from other fast-growing sites?

At this point, MySpace has grown beyond any hosting provider's past knowledge. On the other hand, right now this minute, the founder of tomorrow's MySpace could be ordering his first server at your data center. Ask him what he's up to. Do some research on whether any of your current customers are building similar applications, and what kind of growing pains they've gone through. Turn your hosting company into a repository of knowledge. Because it's the one card you can to play against infinite storage, CPU power and bandwidth.

Matt Mullenweg from WordPress caused quite a stir last week when he vented on his blog about his frustrating experience with Sun's Startup Essentials Program. He wrapped up the post by saying he's "far more excited about what Amazon is doing these days."

Within 23 minutes, Amazon evangelist Jeff Barr - who was recovering from surgery - commented on Matt's blog that he would love to talk. And about a day later, Sun CEO Jonathan Schwartz posted an apology on his blog:

"All I can say is... I'm really sorry, Matt. That's not the way Startup Essentials is supposed to work. We screwed up, and you're completely right to suggest if that's the norm, we should kiss goodbye our aspirations of reestablishing our business in the startup community. If there's anything I can do to win a second chance, I'd like to know."

I totally agree with Red Monk analyst James Governor's assessment of the situation: "the sales battle for contracts, from major to minor, is increasingly being fought out in the blogosphere. If you don't play then you may miss out on a pay day... Go read Matt's blog again. As an RFP." (RFP = request for proposal.)

Now that Amazon and Sun have responded to Matt's RFP, another company that I wish Matt had heard from is LayeredTech. Back in November, Matt mentioned that WordPress has 50+ servers at Layered. Considering (a) the existing relationship, (b) Matt's interest in Amazon's S3 and EC2 services, and (c) Layered's new on-demand, pay-as-you-grow computing service, I'm disappointed that Todd from Layered hasn't joined in the conversation. Especially since both Matt and Stephen O'Grady, James' Red Monk colleague, are both getting interested in Media Temple's GRID platform

Speaking of which, Media Temple's recent blog post really helped spread the world. Along with Matt and Stephen and dozens of others, Scott Yang over at HostingFu is talking about it too. Unfortunately, Media Temple hasn't followed up on these threads either - at least not publicly.

I know that every web hosting CEO is overwhelmingly busy - but I'm sure Jonathan Schwartz' to do list isn't any shorter than yours?? PLEASE... do a Google Blogsearch for your company - then subscribe to the RSS feed for new search results, and keep on top of these discussions. Publicly.

The last company I'm going to pick on in this post is ServerBeach. Kevin Burton wrote last November that "while I like Serverbeach they need to get their act together fast or I might just switch to EC2." I've forwarded this post to at least 3 different people at Peer 1, but the only public response Kevin's gotten has been from... Jeff Barr over at Amazon.

I've been having a lot of conversations - with the 3tera folks, the Tier 1 Research guys and several hosting providers - about how interested enterprise IT managers are in outsourcing their data center requirements.

3tera CEO Vlad Miloushev thinks as many as 90% of all servers are hosted in-house, and research from AFCOM and Gartner both show that a significant proportion of corporate data centers will become obsolete in the very near future. In which case, an attractive market should be opening up for external hosting providers, right? Because surely outsourcing fees will be easier on enterprise CFOs' eyes than construction costs for new data centers?

But blade consultant Martin MacLeod offers some interesting perspective on why enterprises may find outsourcing less affordable than we think. (A server manager he recently spoke with was "horrified" by quotes from colo vendors.)

1. Commercial hosting facilities have higher security standards than internal data centers - not to mention more reliable connectivity, newer power and cooling equipment, etc. What business value does an enterprise IT manager receive from such amenities? That might not be something he can instantly quantify.

2. In fact, speaking of quantifying value, Martin says the "hosting/support fee" within enterprises is absorbed into generic IT expenses. So companies might not have neatly broken down cost per server/cabinet/square foot stats for use in making apples-to-apples comparison with outsourced alternatives.

3. In a November 2006 survey of 500 Silicon.com readers, 33% had IT equipment between 5-10 years old, and 32% had "fully functioning" hardware that's more than 10 years old. According to Sun, servers are improving at a rate of 40% per year in terms of power efficiency, so the difference between power bills for colo-ing pre-historic versus state-of-the-art machines will widen dramatically with time. This represents a dilemma for enterprise IT managers: should they ditch still-functional gear and migrate to brand new equipment? That sounds like an expensive proposition...

Still, Martin agrees that enterprises shouldn't be in the data center building/running business. He says they should outsourced hardware in addition to facilities requirements - and run their virtualized infrastructure on an on-demand service. According to the folks at Dr Dobbs, this is where Amazon EC2 comes in.

If you don't like their conclusion, you should look into offering virtualized utility computing. A few days ago Rich Lee from Hosted Solutions commented on another post that the compute side of on-demand infrastructure is "clearly the most complex piece to integrate, provision and bill properly for", but Hosted is working on addressing this challenge. I hope that Rich and others proceed with urgency, because the window of enterprise outsourcing opportunity won't last forever.

1.Robin Miller was telling me about his new video production service. It started as an experiment; now it's taken off as a business. The challenge is, which should he put the videos he's produced? Free hosting services don't offer high-enough-quality encoding, shared hosting isn't reliable enough, and he certainly doesn't need a whole dedicated server. The solution that came to mind was Amazon's S3. Since most of his mini-commercials are unlikely to be viewed by very large audiences, pay per use makes sense - even if Amazon's $0.20 per GB data transfer is about 100x more expensive than some shared hosting plans.

2. The folks at Dr. Dobb's wrote a long article about using Amazon EC2 in conjunction with Oracle SOA Suite 10g. (It's a set of standards-based components for developing reusable, interconnecting enterprise software modules.) They found EC2 to be "an ideal hosting environment for commodity SOA components". It ushers in "a new era in which reliable, resilient, scalable and high performance SOA deployments can live outside corporate boundaries."

3. Stephen O'Grady from Red Monk, an analyst firm, says Amazon is his pick for 2006's top technical innovator. Amazon "addresses a sizable and growing market" - and what it's doing is "a big deal". It levels the playing field for web app start-ups:

"Many of them are very, very good... but networking, hardware, etc require capital investments that can be onerous, and it's a simple fact that they don't benefit from Google's or Yahoo's economy of scale. They buy hardware piecemeal, and get run of the mill bandwidth deals, all of which adds up to high entry costs."

But Amazon, Stephen says, changes the game completely. And we've only seen the very tip of its iceberg - the more you use it, the more you'll figure out new ways to use it. Maybe tomorrow's departmental servers will be Amazon powered!

---

I know - it just doesn't seem fair. After all, S3 was down last week, while your service probably wasn't. Nonetheless, Amazon has firmly positioned itself as *THE* on-demand hosting solution in the minds of these and other thought leaders - by offering many, many, many use cases. With ongoing updates. In fact, let's set expert opinions aside. Even my friends who know next to nothing about hosting have heard that Smugmug saved $500,000 with S3 and thinks Amazon is the Holy Grail.

In contrast, after talking to Robin, I spent some time trying to think of specific scenarios for which I would without a doubt recommend one particular hosting provider over any other competitor.

Rackspace was on my list, of course. That's where you go when you have a sizable budget and want great support.

Delaware.net is another good example, with its home grown SaaS apps.

And Softlayer's gigabit backend network is great for private server-to-server connectivity. A few months ago, CEO Lance Crosby told me about a company who put all of its franchisers on Softlayer servers and had them communicate with each other and corporate through Softlayer's LAN. See? Case studies really work. Like the Smugmug legend, that example has really stuck in my mind.

Can you think of any other hosting companies that have specifically and uniquely captured any particular "can't get it elsewhere" concept? If you feel your company belongs in this category, maybe you should flood the world with a bunch of success stories.

That's the title of an October 2005 Gartner report. Lydia Leong wrote about a promising market that web hosting providers hadn't (and still haven't) addressed. Large enterprises, she said, often have small projects that don't require industrial-strength managed hosting on stand-alone servers - yet they hesitate to sign up for mass market shared hosting plans that don't offer enterprise-grade redundancy, scalability and accountability.

Lydia recommended that web hosts set up virtualized infrastructure with managed-hosting-like support and SLAs. The goal is to allow enterprises to purchase resources on a cost-per-VE basis, with utility pricing for variable bandwidth needs. A content distribution network, she suggested, might make a good add-on for this product.

Since small projects can grow into large ones (example: Hostway's 100+ server relationship with Fox News began through a $20 shared hosting plan) - and multiple small requirements can add up to a sizable contract, hosting providers who don't offer such a flexible entry point might lose valuable opportunities to build profitable relationships with large enterprises.

I thought of Lydia's report when I read Martin MacLeod's BladeWatch post on cost allocation in an enterprise IT environment. If the IT department has a 350-blade grid, and Application A uses 70 blades worth of resources 15% of the time, should its owner be responsible for 15% of 20% of the cost? What about depreciation? In a separate post, Martin asks whether IT needs to have its own accountants, the better to take each server and split up associated costs per virtual CPU, per virtual GB in RAM and storage... It might take months - at hundreds of dollars per person, per day - to develop processes, procedures and documentation on internal pricing.

My immediate reaction was, wouldn't it be less trouble if they outsourced? AFCOM says 50% of corporate data centers won't have enough power/cooling capacity by 2010 anyway; Gartner thinks this will happen by 2008. The question is, whom should they outsource to? I'm not seeing much discussion of cost-per-VE/utility pricing on managed hosting companies' websites - or enterprise-class support capability from mass market providers. It seems as an industry, we haven't taken Lydia's advice. And maybe we should: doesn't enterprise outsourcing sound like a promising opportunity to you?

IDC, by the way, says we're entering the age of "virtualization 2.0" in which virtual appliances (software packaged into virtual machines; one example is Amazon's EC2 Machine Images) will become household words. I'm a little worried that the web hosting household might not be on the same side of the tracks...

PS - I read about IDC's predictions on Kimbro Staken's Virtualization Daily blog. Kimbro is the CTO of JumpBox; they make virtual appliances.

PPS - Hosted Solutions just issued a press release about - among other things - the importance of "IT as a service". That's way cool - *except* I see only colocation and dedicated servers on their website, no "shared, highly available infrastructure".

Amazon's S3 storage service had a bit of a hiccup last Thursday and Friday. I came across this discussion thread on Amazon's developer forum via CNet. Rich Miller and Dan Farber have also mentioned the incident.

As frustrated as customers must have been by two days of much higher than usual latency and unpredictable errors, their reactions were surprisingly mild. And once Amazon pinpointed the problem (a batch of defective new hardware), the complaints stopped. In addition, three posters thanked Amazon for being transparent about sharing specific details.

In contrast, I've also had the misfortune of putting customers on new hardware that turned out to be faulty, and I most certainly didn't have the benefit of such patience and understanding. Why? The answer is in this video:

Click To Play

If you were making a 5 minute presentation on your company, how would you spend that time? You'd show lots of data center photos. You'd talk about 24/7 tech support. You might fill a slide or two with customer logos - but you wouldn't focus on what cool apps your users have built. Because chances are, you have no clue.

Jeff Barr's speech shows that Amazon is different. Have you ever heard of TV Mojo, he asks? They rock. What about SmugMug? They're so awesome that Jeff himself stores thousands of photos on their site. He succeeds in portraying Amazon as not just an ordinary vendor, but a collaborator in one exciting adventure after another. People sign up for S3 and EC2 not just for hosting, but to be where the action is. As Motorola CEO Ed Zander puts it, what customers want to buy are "cool experiences".

So don't let any technical problems Amazon might have convince you that you offer better web hosting. You're not on a two-dimensional playing field where "no glitches" is all it takes to be "better". Amazon will learn from any technical issues it comes across; it will get better. You may already have solved those challenges; you've been in the hosting market for much longer. But do your services have half the sizzle of theirs? And what are you doing about that??

A friend sent me some compelling-sounding product info on a certain utility computing solution. Check it out:

Traditionally all layers of computing environments have been static, configured (usually manually) to support a single computing solution. For example, hardware is assigned for specific uses (web server or database); the OS is tied to the hardware and storage is designated to specific locations. Applications are installed to run inside this specific, static environment. There are significant testing requirements since all layers of this environment have to be hard-coded to work with one another. The result is a tightly bound configuration that does not adapt well to changes in business needs or to the introduction of new and updated applications.

Instead, this vendor says, it makes sense to move to a virtualized utility computing environment.

This vendor is Softricity, which Microsoft acquired last May. My immediate reaction was, will Microsoft use some of the data center space it's building/planning to offer a hosted version of Softricity? Office Live today, SoftGrid Live tomorrow...

Earlier today, Tier 1 Research analyst Dan Golding mentioned in an interview with the WHIR that "there isn't an industry in the world where there are more than four or five major competitors". He was talking about CDNs, but Sun's Greg Matter agrees that the same might apply to Internet computing.

Microsoft quite obviously intends to be one of the 5 Big Computers. Chief Software Architect Ray Ozzie is all about the software to services transition - and he's got a $980 million budget for just one data center. Will one of the remaining 4 spots be yours?

During Tier 1's hosting conference last fall, I was intrigued with CEO Andy Schroepfer's recommendation that business hosting providers consider partnering with Microsoft Live. That raises a very good question for dedicated server providers whose core business is wholesaling to shared hosting resellers: what's on the next chapter of your business plan?

I've been reading "Why Not?", a book about innovation. One of its key concepts is to ask yourself what Croesus (the ancient rich king) would do. In other words, if you could throw unlimited amounts of money at a problem, what solution would you pick?

Web-hosting-wise (unless you're Google), maybe a more practical approach is to ask what Robert Marsh would do. As founder/Head Surfer of EV1, Robert single-handedly created the discount dedicated servers market. He also popularized complex hosting among EV1 customers by introducing private racks, and rolled out VPSes with a BIG party, complete with fireworks.

He blogged (sort of) before blogging was fashionable (by making a personal soapbox out of his customer forum). He saw potential in APC's Mobile Data Center before Sun made a splash with the Blackbox. He wanted to travel regularly to cities with high customer density - much like what Amazon Web Services evangelists are doing. And he put some work into developing a beyond-the-box hosting environment, which so many hosting companies began offering last year.

I've compiled a quick collection of facts and stats that I think will define web hosting in 2007. (If you're reading in RSS and the slides don't show, click here.) If Robert were still in the hosting business, what would he do in today's market?

Would he build his own S3? At least two Rackspace customers have traded their managed storage for Amazon's pay-per-GB solution - and just this morning my new friend Santosh asked which hosting companies offer S3-like shared storage. What should I tell him??

Might he take Tier 1 Research analyst Dan Golding's advice and snap up a start-up CDN? Buy a shipyard and offer container colo for Sun Blackbox owners?

He'd have another Birthday Bash, that's for sure. Or several parties in different cities - he'd drive his mobile data center right up to the entrance of not just HostingCon, but Salesforce.com's and VMWare's user conferences. Did you know that 7000+ people attend each? And they work for companies that could become your customers!

The most interesting figure (on slide #18), BTW, comes from Vlad Miloushev of 3tera. Vlad thinks up to 90% of web servers are hosted in-house. Robert wanted to go after this market with not just private racks, but private suites.

What else? Who knows. If I were you, I'd take the guy to Fleming's the next time you're in Houston. He might have some insights for you!