A Brief but Impressive History on eBay's Architecture

Through Rich Miller's post on Data Center Knowledge, I came across this presentation (PDF) on eBay's architecture.

eBay was built on FreeBSD/Apache/Perl/GDBM back in 1995. It ran on hardware that could be bought at Fry's. In 1997, the site migrated to NT/IIS/C++ with Oracle DB on Solaris. In 2002, everything was rewritten in J2EE. Now its system runs on custom developed architecture which - according to Joe Gregorio - looks a lot like Google's BigTable.

eBay runs 15,000 instances of its site across 8 physical data centers. Its 2-petabyte database contains 200x more data than the Library of Congress. Its system consists of 6 million lines of code, with 100,000 more added every other week. Its 212 million users maintain 105 million listings at any given time, generate 1 billion daily pageviews, and conduct $1590 in transactions every second. Its infrastructure supports 100 million API calls and 26 billion SQL executions per day.

Greg Linden, who wrote Amazon's recommendation engine, says he sees a lot of parallels between the evolution of eBay's architecture and Amazon's. Might eBay be next in rolling out an utility computing platform??

Frank Sommers concludes that there's more to eBay's success than technical brilliance: "the key test of scalability is not so much how each architecture stage scales, but how readily a company can move an application from one architecture step to the next. That indicates that scaling is as much an individual or organizational question as a technical one.". Note to hosting providers who feel hobbled by legacy backend systems: there's light at the end of the tunnel. It is possible to move on to the next stage while keeping your services operational.

And Dan Pritchett, an eBay Technical Fellow and the presentation's co-author, has an interesting post on what scalability means: "Is that transactional headroom, storage improvements, reduction in operations staff, or time to market for new features? Can you achieve your 4X scalability if you experience a 4X increase in data volume?" It's a great read for anyone whose customers are asking for scalable hosting solutions.

PS - Fun fact, via HipMojo: eBay's creation myth (which I'd never heard of) was a hoax. The site was NOT built for selling Pez dispensers.