Part of a diagram from the Selenium website illustrating the testing environment
(WEB HOST INDUSTRY REVIEW) — With more people accessing and relying on Web applications every day, their smooth and efficient operation is no longer just an advantage – it’s a necessity.
Designed specifically for testing web applications, Selenium (www.seleniumhq.org) provides a suite of tools that can be used via a Firefox add-on that records clicks, typing, and other actions to make a test that can be played back in the browser, which can help find bottlenecks and other problems.
Jason Huggins, co-creator of Selenium Core, creator of the Selenium IE browser extension (Selenium Ice), has helped introduce Selenium to the power of the cloud, in a product called “Sauce OnDemand.” The new service enables users to run Selenium tests in parallel across multiple browsers in the cloud.
In an email interview with The WHIR, Huggins, co-founder and “executive software chef” of Sauce Labs explains the importance of testing tools in the application development process, the advantages of running tests using cloud computing, and how Sauce OnDemand can be integrated with other services.
WHIR: What were your goals when you began developing what turned out to be the very popular cross-browser testing tool Selenium? Have these goals been met?
Jason Huggins: At a technical level, the goal for the Selenium project is (and was at the beginning) to enable software developers to verify every feature that worked yesterday still works today. At a higher level, Selenium is meant to be a confidence booster. When you see all your tests running in the browser, you feel confident all your new code won’t break anything else. And with that confidence, teams can move quicker and ship software more often.
The goals were met, but the project can’t stand still. Web browsers keep changing and getting better, and developers keep creating new web applications to take advantage of new browser features. The Selenium project (and all testing tools) are in a constant game of catch-up to make sure it stays relevant to software developers and testers.
WHIR: What are the challenges to executing Selenium scripts across the globe? What impact does the use of cloud computing have on running such operations?
JH: When you have lots of Selenium tests, the challenge is to keep the testing process as fast as possible. Most teams only set up one or two machines for running tests. After a few months, as more tests get written, and each single test takes about a minute to run, the testing process can take a long time – 2 to 12 hours is not uncommon. When that happens, developers start to get annoyed at the slow feedback cycle.
Cloud computing makes it possible to dramatically reduce the build and test cycle, by making it easy to procure as many machines as necessary to run all your tests in parallel. Many customers see a 10x speed improvement when they run their tests in parallel on many machines instead of sequentially on one machine. When ran in parallel, the total test cycle is a fraction of the time it takes to run any one test. Of course, this speed-up comes at a price – it takes time and money to manage all these cloud test machines. That’s where Sauce Labs comes in – we give you the speed of faster tests on a large cloud of machines, but we remove the headache of maintaining the infrastructure yourself.
WHIR: How does a company decide whether Selenium or a more robust testing solution like Sauce OnDemand is needed?
JH: It’s a “build vs buy vs rent” decision, with the key scarce resource being your developers’ time. Would you rather have them writing new features for your site or maintaining the infrastructure to keep it running? Most would agree that writing new features is more important. When you add up all the dollars per hour your staff spends setting up and maintaining software development infrastructure, Sauce OnDemand is a great value and time-saver.
WHIR: Are there currently ways for online service providers such as web hosts to incorporate Sauce OnDemand into their offerings? Is there an API or white label version?
JH: If a service provider is focusing on the agile software development process, we’d love to talk to them. We do have an API, and we’re continually publishing how-tos and code for integrating our service with other build and test software. Our most recently published integration is for the Hudson continuous integration server and Sauce OnDemand. We don’t provide a white label service at this time, but I’d never say never.
WHIR: Can you give me an idea of how developers will be able to use “Sauce TV” to get a better view of how their applications run in the cloud?
JH: When you run browser tests in the cloud, instead of on your laptop, you lose the immediate visual confirmation of seeing the tests run. It’s kind of like the old philosophical riddle of the tree falling in the forest. (“does a falling tree make a sound, if you’re not there to hear it?”). By analogy, if you can’t see the browser running your test in the cloud, is it really working? To solve that riddle, Sauce invested in many visual tools to provide a window into our cloud. So now, while a test is running, you can watch it live on Sauce TV – a real-time in-browser remote desktop viewer. We also record a screenshot of every test step, and record of the entire test, all viewable on our website. Sauce TV is like “WebEx” video conferencing, but instead of watching a PowerPoint sales pitch, you’re watching Selenium automate our browsers and checking your site.
WHIR: One of the goals of Sauce TV is to illuminate critical “blind spots” when testing applications in the cloud. Could you tell me about some of the problems that are not found with typical testing solutions?
JH: The key problem that Sauce TV is solving is shortening the feedback cycle. When you write new code, you want to instantly know if that broke anything. Sometimes, it’s a simple thing like an unexpected popup window that causes a test to fail. Being able to quickly see the screen to see what’s happening is a huge boon to the developer. However, when Sauce OnDemand runs tests in the cloud to give you speed, and those tests are run on dozens or hundreds of invisible machines, you’d otherwise have to wait a minute or two for the screenshots and video before you can see if anything went wrong. Instead, Sauce TV gives the developer real-time access into our cloud. Real-time access is especially critical when a developer is debugging a single feature and a single test – a developer might run the test several times in rapid succession. The faster the cycle goes, the better.
WHIR: Is there anything else you would like to put out there about cloud-based testing?
JH: Cloud-based testing is one of the huge success stories for cloud computing. Testing is often the first kind of cloud-based service that companies are willing to try. Sauce has now run over 2 million tests in the cloud, the equivalent of more than 3.5 years worth of testing. Not bad for a company that’s only two years old.
No related posts.
