I came across Uptime Institute founder Ken Brill's CIO Magazine article via 3tera VP Marketing Bert Armijo's blog.

Ken says while hardware prices are falling, total cost of data center ownership is headed through the roof. 5 years from now, the purchase price for a rack of servers will drop 27.5% from $138K today to just $103K. But while it only takes 15 kilowatts to power that rack right now, the energy requirement will rise to 22 - 170 kilowatts by 2012. It could cost as much as $2.3 million to power/cool $103K worth of gear throughout its 3-year lifespan.

(I'm not sure if this figure includes switches and routers and such. A recent Cisco/APC/Emerson study shows that servers/storage/cooling consume 76% of data center power, with 11% going to networking equipment, 3% lighting, and 10% power conversion losses. If Uptime's calculations didn't take the other 24% into account, Ken's $2.3M becomes over $3M!)

I've been thinking about Ken's stats and trying to understand what they mean. As a point of reference, I was looking at Dell's website, which advertises the 4U PowerEdge 6950 dual core, dual processor Opteron server for about $9K. Is Ken saying that:

(a) This particular machine will cost 27.5% less 5 years from now?

(b) 2012's late model machines will sell for 27.5% less than what's on the market today?

(c) The amount of server hardware that fills up 4U of space will be available for $6500 in 2012?

If we assume he means (c), and we accept Sun's claim that "server performance, power and space efficiencies are improving at up to 40% annually on average, and could double every 2 years", then 4U of space may be able to accommodate not one but 4 servers that each feature 4x more processing power and 4x greater energy efficiency.

In other words, $6,500 could buy you 16x more computing resources than that dual Opteron! If that's the case, you might even be able to afford $1M per rack per year in electricity. But only if you virtualize like crazy. No more leasing data center space per square foot or per rack. No more dedicated servers, either. The average customer won't need 4x more processing power in 5 years, which means you won't be able to justify turning on a whole entire server just for them.

You'd also have to replace hardware early and often. Sun recently announced a refresh service for swapping out your servers at least 3 times over 42 months. At first I thought that sounded wasteful, but if server power efficiency is improving at 40% per year, holding on to old gear might end up costing you more. Again, virtualization would be a must. You wouldn't want customer apps to become attached to machines that will be phased out before long.

Bert from 3tera says changes in data center economics will make it increasingly difficult for enterprise CIOs to justify operating their own facilities. But they won't outsource to traditional colo or dedicated server providers. Instead, he agrees with Cassatt CEO Bill Coleman that in the near-ish future, you'll be "paying for data center horsepower the same way you pay for electricity or gas". I think so too. How about you?

PS - On a somewhat related note, eWeek says Intel will release its "Clovertown" chips today. The quad core processors have a 50 watt thermal envelope, versus 80-120 watts on earlier models. That's a 38-60% drop.

PPS - Also, speaking of the Uptime Institute, check out this SearchDataCenter.com interview on how they've helped The Planet save $10K/month on electricity. The Planet, the article says, is looking to expand beyond Texas into the Midwest.

When I first met Raju Vegesna at the Office 2.0 conference last October, Zoho had just passed the 100,000 user mark. Less than 5 months later, Raju says they're closing in on 200,000.

While Nielsen/NetRating (PDF) claims that Google has a 92% share of the web-based productivity apps market, Raju points out that Zoho and ThinkFree together attract as many unique visitors as Google Docs and Spreadsheet. (See Ismael Ghalimi's elaboration on the user count discrepancy.) 432,156 users visited Google Docs last December; that's 3% fewer than in October. Zoho, on the other hand, has undergone rapid growth (as CEO Sridhar Vembu puts it, "we try harder") - and hopes to beat Google to the 1 million user goal.

By the way, when Raju and I were introduced, a colleague of his described him as Zoho's PR strategist. I just found out that in addition to attracting favorable coverage in every possible media outlet, he was also in charge of company's data center infrastructure. (Raju recently managed to offload this responsibility.) He insists it wasn't as much work as it sounds, because Zoho's apps run on a self-healing grid that's attached to a pool of standby servers. If any application server dies, the appropriate code will automatically be deployed on one of the standby machines. The failed hardware can be swapped out whenever someone has time. As for customer data, everything is replicated three times and backed up offsite.

(Zoho's Rackable servers are colo-ed at Savvis and Equinix. They're in the process of migrating from 32 bit to 64 bit machines with dual core Intel Woodcrest processors and 8 to 16 GB of RAM. The new machines arrived last week, but won't be deployed until late March. As Google's recent hard drive study shows, a longer burn-in period can help weed out faulty hardware.)

Zoho's HR processes are similarly efficient. Adventnet, Zoho's parent company, has 600 employees (and counting). Whenever a new Zoho product is conceived, a dedicated team of 2-6 developers is allocated to the project. Having rapid access to talent gives Zoho a tremendous competitive advantage. Forget Google Office; the loooong list of ideas and partnerships that Zoho's in the process of implementing might coalesce into Zoho Life: single sign on access to an ever growing suite of fully integrated web apps that do... everything.

Keep an eye on Zoho; they're setting the standards for delivering a seamless SaaS experience. And sign up for an account if you don't already have one. Lastly, stay tuned. They've got some big announcements on the way.

When Gary Chaffin launched Stargate 12 years ago, his core business was domain registration. The company got into shared hosting in the late 1990s and dedicated servers in 2001. As Stargate's customer base grew, so did its data center infrastructure. Gary built out his Naperville, IL facility with APC InfraStruXure power, cooling and rack components.

APC's chilled water cooling distribution units are designed for raised floor environments, which Gary wanted to avoid. As APC CTO Neil Rasmussen puts it in this SearchDataCenter article:

"I can make an air conditioner any size I want, but the problem is getting the air through the tiles. If I try to moving 25kW of air through tile, it's going to be 120 mph coming through floor tile. It's also very inefficient to push all that air over a distance. It takes a tremendous amount of horsepower to move it around. It's not uncommon to find just the fan taking more power than the servers in data centers."

Rasmussen's energy-saving strategy is cabinet level cooling, but Gary (who'd spent many years in the medical diagnostics industry, where he had extensive experience with MRI heat dissipation) came up with a different solution. With APC's assistance, he redesigned its hot aisle containment system to eliminate the raised floor requirement.

APC described Stargate's custom install to other customers; based on their overwhelming interest, Gary decided to build a second data center. The facility (which is schedule for August, 2007 completion) will offer 10- to 35-rack "vaults" with no mix of head loads; customers will have access to precise data on UPS run time, power utilization, etc. for their isolated environments. Since January, over 20 large enterprises, including several financial institutions, have stopped by for tours.

Stargate picked a great time to get into the data center business; there's a tremendous market demand for space. But support-wise, how will the company handle its increasingly diverse customer base, ranging from individual domain owners to large banks? With different staff, said Gary - seeming surprised that such a minor detail could be considered an obstacle.

BTW, Stargate will be sharing its progress through its new blog. I wonder what's next for Gary, now that he's exhausted the entire range of hosting possibilities. We did talk briefly about Sun's data center in a box and APC's mobile data center. You never know; Stargate might find itself in the "container colo" (or data center truck stop) business one of these days.

Dr. Jonathan Koomey, a staff scientist at the Lawrence Berkeley National Laboratory, says that US servers consumed more electricity in 2005 than all Mississippians combined. I read about his AMD-sponsored study on CNET. It reminded me of Nick Carr's calculation that a Second Life avatar uses more power than the average Brazilian.

In his LinuxWorld keynote, AMD's Randy Allen says this is the first time that national and global data center energy usage has been comprehensively analyzed:

* In the US alone, total 2005 usage was 45 billion kWh, resulting in utility bills amounting to $2.7 billion. Worldwide consumption was 123 billion kWh, or $7.2 billion. This represents double the electricity demand in 2000.

* Based on IDC's server shipment forecasts, server-related power usage will increase another 40% by 2010 - if per-server power consumption stays at 2005 levels. But if server power demand grows at past rates, we'll face a 75% increase by 2010.

Dr. Koomey's assessment made me think of Sun's claim that "server performance, power, and space efficiencies are improving at up to 40 percent annually on average and could double every two years". I wonder at what point Koomey's "past growth" trends ended and the efficiency improvements Sun mentioned began?

By the way, Koomey's stats take into account only servers and "supporting infrastructure", such as cooling and lighting. Networking and storage equipment are excluded; Koomey says their power consumption is 1/3 that of servers. (So total = 75% servers + cooling, and 25% networking + storage?) This isn't quite consistent with Cisco's breakdown of 50% cooling, 26% servers + storage, 11% networking and 13% miscellaneous. Or HP's finding that 60% of data center power goes into cooling.

Also interestingly, San Francisco Chronicle staff writer Ben Pimentel isn't sure that report's findings are as alarming as AMD makes them out to be. After all, server related electricity consumption was only 1.2% of total US and 0.8% of total world energy usage. Quoting California Energy Commission assistant executive director Claudia Chandler, he says it's an urban myth that the Internet's growth will cause an energy crisis. (This Wired article for instance, mentions 1999 projections by energy analysts Peter Huber and Mark Mills that data centers will consume half of all the world's output of electricity by the end of this decade; now that does seem like an unlikely story.)

But let's look beyond how these numbers add up and try to answer Allan Leinwald's question. We're making ongoing investments on acquiring and operating more, bigger, better servers - but how will we use their computing resources to build new and improved infrastructure and services? Allan points out, for instance, that in the relatively near future, Intel's 80-core processor will be out on the market for a few thousand dollars. Surely they're good for more than the same old unmanaged dedicated servers?

PS - Check out Nick Carr's post on the Koomey report. Two important points: 1.2% of national power consumption is a comparable amount to usage by color TVs. Also, custom built servers, such as those in Google's data centers, weren't considered for the study.

VeriSign CEO Stratton Sclavos announced during his RSA 2007 keynote that the company will invest over $100 million to increase its DNS query capacity from 400 billion to 4 trillion per day. The initiative is code named Project Titan. Rich Miller at Data Center Knowledge reports that it will expand VeriSign's presence from 20 regional centers around the world to 100+, and scale connectivity to its resolutions systems from 20Gbps to 200Gbps.

The interesting thing is, VeriSign currently handles just 24 billion queries per day, which means it should have available capacity for 376 billion additional queries? And according to the New York Times, VeriSign expects the Internet user population to increase no more than 2x (from 1 billion to 1.8 billion) by Project Titan's completion in 2010. But ZDNet's Dan Farber says VeriSign's DNS infrastructure will also have to support 2 billion cell phone/PDAs, 63 million IPTV users and 34 million VoIP households by 2010.

In other news, the nation of Hackistan claims responsibility for Tuesday's attack which nearly took down 3 of the Internet's 13 root servers (VeriSign operates "A" and "J").

Well, ok. Not really. Lovely Hackistan (which I read about on Business 2.0's Dawn Patrol blog) is a figment of Fortify Software's imagination. The really funny marketing campaign drew huge crowds at Fortify's RSA booth. In addition, it's inspired a 150% increase in attacks on Fortify's website. You should check it out!

During an early 2004 customer gathering, former EV1 CEO Robert Marsh asked attendees whether they'd be interested in putting multiple servers on the same private network. I still remember how their eyes lit up. 100Mbps server-to-server connectivity? Oooh yeah!

I had a similar reaction to this announcement on SoftLayer's new "customer exchange". SoftLayer says every single one of its dedicated servers is now deployed with a secure gigabit connection into its private Meet Me Room. Imagine scenarios like Zoho's integration with OmniDrive, or tools from Flickr, del.icio.us, etc that automatically output users' content on their TypePad blogs. SoftLayer customers can now jointly develop these sorts of cross-site interactivity without incurring bandwidth charges for data transfer between their systems. How cool is that?? (Better yet, SoftLayer actually saves money by keeping traffic between customers within its private network instead of paying for packets to travel in and out of its data center.)

Earlier this week, AlertLogic CTO Misha Govshteyn and I had a discussion on customer service via his blog. I think we came to the agreement that while investments on delivering an exceptional customer experience can pay off big time, your ROI is contingent upon having an outstanding product (which itself is a key part of the customer experience). SoftLayer is a great example of how hosting providers can combine both.

By the way, SoftLayer's mobile portal is another customer-friendly new feature. In addition, the company recently launched a new data center with 500 watts psf of power capacity. It also upgraded its Verio and Internap uplinks to 10GigE.

I hope SoftLayer's upcoming projects include an application directory where customers can find partners for creating mashups with. (I think this could become a powerful sales tool as well.)

I read about it on CNet. What you see above is Lucasfilm's data center. Its network features 300 10-gig ports and 1,500 1-gig ports. Every digital artist within the company has a desktop with gigabit connectivity.

Lucasfilm uses Verari servers with dual core, dual Opteron processors and 16 GB RAM as well as bunches of legacy machines. IT director Kevin Clark says it can take as little as 6-7 months before new equipment turns legacy. For storage, Kevin has 300 TB on a NetApp NAS. (BTW, check out this Infoworld article about NetApp's Data Ontap GX virtualization software, which allowed Lucasfilm to maintain 200 TB across 20 servers while having storage capacity appear as one single 200 TB disk. Lucasfilm has been testing the software for the past year, and will be moving to a GX cluster soon.)

As with Google, Lucasfilms says power utilization is an important factor in its data center equipment selection. The company's website also points out that its facility is LEED certified by the US Green Building Council. I've been hearing a lot about LEED these days. It's mentioned in this SearchDataCenters article, and it will be on the agenda at Data Center World. If green IT is a priority for corporate data center managers, it will likely become an increasingly important topic in your sales discussions with enterprise customers.

CNet reports that Google has started placing "high volume" orders with Intel (red in chart below). That's unhappy news for AMD (blue), especially since Intel has also barged in on the exclusive relationship it's had with Sun since 2005. Here's a comparison of changes in the two company's share prices over the past month:

Considering Google's enormous data center projects in Oregon, North and South Carolina and within NASA's Ames Research Park, can you imagine what "high volume" means?

While the Sun deal required Intel to make certain commitments to Solaris, Google didn't ask Intel to endorse the non-existent-for-now GoogleOS. But Intel did have to develop unique mother board/memory module designs to meet Google's power, cooling, performance and cost requirements.

BTW, the CNet article offered some interesting data points from IDC analyst Michelle Baily:

1. New server spending is increasing at about 2-3% per year, but power/cooling costs are growing 4x faster.

2. For every $1 spent on new hardware purchases, companies spend 50 cents on power and cooling for their installed base.

(Baily's stats are oddly inconsistent with her IDC colleague Vernon Turner's report that the server market grew by 2% in Q3, 2006 alone. Gartner is even more optimistic; research director Jeff Hewitt told CRN last November that Q3 server growth was 9.1%. Go figure...)

Robert Cringely says Google is on its way to become the biggest CDN ever. It controls more network fiber than any other company in the world - and it's building two huge data centers in South Carolina alone. Google's enormous capacity, coupled with end users' increasingly voracious appetite for all things web-based, will lead to a future in which Google will act as a giant proxy server for the Internet. It will be "our phone company, our cable company, our stereo system and our digital video recorder"...

Greg Linden disagrees. He thinks that instead of cornering the market on bandwidth, Google is trying to build a world of infinite storage and CPU power. Well, *and* bandwidth.

No web hosting provider will have access to Google's economy of scale. Or out-of-this-world local government incentives. (Unless you're a state-licensed operator in China, but that's a different story.) Sorry, there isn't and won't be any way around this. BUT one resource that Google won't be dishing out in buckets is... knowledge.

I came to this realization while reading this article on MySpace's infrastructure. Once again, I spotted it on Data Center Knowledge!

MySpace was originally built on Perl + Apache + MySQL, but before the site went live, its developers switched to Windows + Cold Fusion + MS SQL. Later on everything was rewritten in C# + ASP.NET. The site started out with just one database, then write vs read transactions were split between one master DB and two slaves. Soon this evolved into vertical partitioning, or a separate database for each and every feature. Finally its technical team settled on running separate database instances for each block of 1 million accounts. All logins come through one single front-end DB, which redirects each user to the database containing files associated with his account.

In the beginning MySpace had server-based storage, then it built a storage area network with room for more disk drives. At one point the company had two full time technicians (!) manually distributing SAN resources between database instances. Finally it moved on to a 3PARData's virtualized storage solution, on which all disk drives can be accessed as one single pool of capacity. In 2007, MySpace plans to replicate its SAN (which is currently in LA) in two other locations, to eliminate its dependency on one single data center location.

Other major changes MySpace has made to its infrastructure include a caching layer between its web and database servers. In addition to minimizing DB lookups, machines on the caching tier are used to store temporary session data, which aren't given permanent database space. Also, Myspace was among the earliest adopters of MS SQL Server 2005, the better to take advantage of its 64 bit support. The 32-bit SQL 2000 limited MySpace to 4GB RAM per server. Now its standard config is 64 GB RAM.

Throughout endless trials and errors, MySpace's developers were under enormous pressure to keep its service online. Could they have benefited from having a team of Internet infrastructure experts to bounce ideas off of? Comparing their experience with benchmarking data from other fast-growing sites?

At this point, MySpace has grown beyond any hosting provider's past knowledge. On the other hand, right now this minute, the founder of tomorrow's MySpace could be ordering his first server at your data center. Ask him what he's up to. Do some research on whether any of your current customers are building similar applications, and what kind of growing pains they've gone through. Turn your hosting company into a repository of knowledge. Because it's the one card you can to play against infinite storage, CPU power and bandwidth.

I read about it on ValleyWag: Google is going to build a $600 million data center in Lenoir, North Carolina. WRAL reports that Google will receive $100 million in tax benefits over the next 30 years. That's $500K for each of the 200 jobs that the data center is expected to create.

WRAL says North Carolina was going up against South Carolina for the Google project. Does that mean the rumored $750 million investment by "Maguro Enterprises" near the South Carolina coast isn't related to Google? Or has the "high tech development project that needs access to lots of electricity" been shelved? (Update from Rich Miller at Data Center Knowledge: South Carolina plans have NOT been put on hold. In addition to $600 million in North Carolina, Google plans to spend $1 billion+ on TWO facilities on South Carolina.)

By the way, as a point of comparison, Rich says Microsoft's tax abatement deal from San Antonio totals only $20.7 million. Microsoft plans to invest at least $550 million in the 470,000 square foot data center project; the facility is expected to create 75 jobs.

Speaking of data center locations, check out Pingdom's where-not-to-keep-servers map. I found out about that through Data Center Knowledge as well. Thanks, Rich!