Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. March 2009 Volume 1 SOFTWARE TESTING New software testing technologies bring new challenges • CHAPTERTEST VIRTUAL 1 LABS MAKE MOST OF CLOUD • CHAPTERRIAS TESTING 2 FOR SECURITY HOLES
  2. 2. CHAPTER 1 1 Virtual test labs make most of cloud QA and testing organizations are embracing the powers of virtualization and cloud-based computing to perform simpler testing at lower costs. BY COLLEEN FRYE a EDITOR’S LETTER p VIRTUALIZED TEST LABS can make she said. “This means that software a it faster and less expensive to set up can languish without being tested due CHAPTER 1 VIRTUAL and tear down test configurations, to poor resource allocation and man- TEST LABS MAKE MOST better utilize resources, and help to agement.” OF CLOUD boost overall software quality. And This bottleneck can be costly, Bal- now cloud computing brings the lou said—an issue that becomes even promise of even less infrastructure more critical in a down economy. “Vir- a to worry about, which can help to fur- tual test lab management can help CHAPTER 2 TESTING RIAs ther alleviate quality assurance (QA) address these kinds of issues, as well FOR SECURITY HOLES bottlenecks due to resource issues. as cutting the cost of infrastructure “QA and testing organizations are by augmenting or replacing physical increasingly embracing virtual test systems.” labs due to the efficiency and cost With the emergence of cloud-based savings that can result,” said Melinda- offerings, the benefits of virtualization Carol Ballou, program director for can be more immediate, Ballou said, application lifecycle management at “because you don’t have to configure IDC. According to Ballou, one of the the software.” barriers to QA is limited access to— and poor management of—physical infrastructure for test labs, as well as STARTUP GAINS CAPACITY, the effort involved in setting up and SAVINGS WITH VIRTUAL TEST managing systems configurations for LAB SOLUTION the labs. For startup Apptio, a provider of “Groups tend to hoard the physical on-demand IT cost transparency resources so that they will have solutions, a virtualized infrastructure access to them when needed, even provided both affordability and the when there is no immediate demand,” capacity to test more broadly. 4 SOFTWARE TESTING MARCH 2009
  3. 3. According to Colin Henry, senior soft- Seattle-based Skytap describes ware engineer at the Bellevue, Wash.- itself as “serving up virtual machines based company, Apptio is developing over the Internet.” According to CEO a data integration add-on for its pri- Scott Roza, “There is no infrastruc- mary product; the add-on will aggre- ture, no upfront expenditures, and it’s gate data from disparate data delivered 100% as a service.” Skytap sources. Virtual Lab “is a multitenant model,” “We realized this portion of the product covered a broad range of systems—ERP, database, etc.—and “Being a startup, we the setup/teardown time for that don’t want to spend the would be costly,” Henry explained. world on getting three a “We knew we needed a virtual infra- EDITOR’S LETTER structure. Being a startup, we don’t racks of servers or a want to spend the world on getting huge box to throw five three racks of servers or a huge box or six VMs on. I knew a to throw five or six [virtual machines] we wanted to go for CHAPTER 1 VIRTUAL on. I knew we wanted to go for virtual- TEST LABS ized space hosted on the cloud.” virtualized space MAKE MOST OF CLOUD Henry said Apptio had performed hosted on the cloud.” manual testing on its primary product —COLIN HENRY and did not have a traditional testing Senior Software Engineer, Apptio a environment, “so we were starting CHAPTER 2 TESTING RIAs from scratch.” FOR SECURITY HOLES The company researched on- he said. “All you need to access it is premise and hosted virtual test labs, a browser. With the browser you can “but any kind of managed service on get access to computing resources; a local machine takes a lot of your you’re not limited by the number of administration time,” he said. Provi- VMs you can run or the number of sioning a box can take three to four users you can have accessing [the hours, according to Henry, and if environment].” you’re adding application infrastruc- While QA and testing organizations ture on top of that, it could take up have been early adopters of virtualiza- to eight. So Apptio decided that tion, Roza said, “very few QA organi- newcomer Skytap Inc. with its Skytap zations are all virtual” because “large Virtual Lab product fit the bill. The database servers and databases his- result? torically don’t work all that well in a “Going from 12 hours to five min- virtualized environment.” He said Sky- utes,” Henry said. “That’s a huge tap offers a hybrid model. “We have increase in productivity, because secure bidirectional VPN technology, you’re not slogged down in installing so you can build part of your lab in the software.” cloud and connect back to your own 5 SOFTWARE TESTING MARCH 2009
  4. 4. data center. The hybrid model can merly VQMS) automates the deploy- take advantage of the cloud as an ment, configuration and teardown of extension of an in-house lab.” complex software environments. Mal- Roza likens the Skytap service to colm said to get started with virtual- a family cell phone plan. Henry said ization, QA organizations need the Apptio has a plan for about 2,000 physical infrastructure to support a hours per month. “We don’t have to virtual lab, a hypervisor platform like keep the infrastructure running full- Microsoft’s or VMware’s, and test time,” Henry said, “so we’re not using automation tools. Surgient integrates unnecessary hours of their time. with HP’s quality management suite When you’re done, you’re done. To and IBM Rational. “The testing tools get rid of a VM you’re not using any- will make the calls to the Surgient a more, you just delete and within a few platform to automate test configura- EDITOR’S LETTER minutes it’s gone from the virtual lab. tions,” he explained. You can manage it relatively easily.” Surgient customers can choose a Roza added that pioneer Software hosted offering, but most install on- a as a Service (SaaS) companies like premise, Malcolm said, and build CHAPTER 1 VIRTUAL broke down barriers what he called an “internal or private TEST LABS MAKE MOST of concern that SaaS might not be cloud.” OF CLOUD secure. For Henry, with their test cases, “we’re not dealing with sensi- tive data, so in that regard it’s not VIRTUAL TEST LABS a much of an issue. For long-lead VMs IMPROVE UTILIZATION CHAPTER 2 TESTING RIAs left on for monthly tests we have it Insuresoft, a software provider for FOR SECURITY HOLES set up over the VPN with Skytap.” the property and casualty insurance industry based in University Park, Ill., decided to install Surgient’s QA/Test AUTOMATION EASES Solution on premise. “We wanted the TEST CONFIGURATION PROCESS flexibility to do what we wanted with But testing organizations don’t need our images and be able to trouble- a cloud-based service to reap benefits shoot ourselves and go through the from virtualization, particularly for process of learning about virtualiza- setting up and tearing down test con- tion,” said Hemanth Guttikonda, figurations. “Automated testing took a Insuresoft’s QA manager. “We had lot of labor out of the process [before some expertise in-house to do some virtualization], but there was still a lot of these things; also we’re not a large labor involved in setting up and tear- enough organization to consider ing down the configurations,” said hosting.” Dave Malcolm, chief technology offi- Guttikonda said Insuresoft turned cer of Surgient Inc., an early pioneer to virtualization to help with testing in the virtualization space. consistency first and foremost, and Surgient’s QA/Test Solution (for- the company ultimately benefited 6 SOFTWARE TESTING MARCH 2009
  5. 5. from better resource utilization as way up.” well. He said the testing lab prior to Consistency and quality are up as Surgient was somewhat ad hoc, and well, he said. “Now people see the they would set up and tear down soft- benefits of how quickly we can get the ware as new versions of their Dia- environment available and ready for mond product, a hosted application, testing. In the past it sometimes took needed to be tested. “We have 15 to 20 companies we service and have physical hardware “People are starting for each company. We had about to understand that 30 to 40 physical servers, which mir- rored our production environment.” [with Surgient] we’re a The process of installing and unin- actually testing in EDITOR’S LETTER stalling software to test for each a true production customer was tedious, he said. environment.” While some testing was done in a the lab, Guttikonda said other testing —HEMANTH GUTTIKONDA CHAPTER 1 QA Manager, Insuresoft VIRTUAL was done on individuals’ physical TEST LABS MAKE MOST machines, which he said wasn’t mim- OF CLOUD icking what the product environment a day or more to get the testing envi- looked like. “The technology we were ronment set up, so people would developing wasn’t being fully tested.” abandon the idea of a central environ- a Insuresoft was already utilizing ment.” Once the organization started CHAPTER 2 TESTING RIAs automated testing tools from HP and using Surgient, “we started noticing FOR SECURITY HOLES saw Surgient, which supports HP, as that we were finding more defects.” a logical choice, he said. These were defects that weren’t Guttikonda said the company’s test showing up when testing on local lab has been reduced from 30 to 10 machines, he said. “People are start- servers. “In essence, we trimmed our ing to understand that [with Surgient] physical servers by over 60%, and we’re actually testing in a true pro- we’re getting more utilization out of duction environment.” the 10 servers than with the 30 in the In addition to getting used to the past,” he said. “But it really means cultural change, Insuresoft had to much more—it’s the ability to tear invest in more powerful servers than down the environment and reuse the what Guttikonda called the “entry- servers.” Previously, he said, each level” ones they had in order to get server was dedicated to one customer the performance benefits of Surgient. and couldn’t be used for any other However, Insuresoft is already seeing purpose. “Now we can use [the serv- the cost savings of better resource er] for one company, and when we’re utilization. “These servers are about done, tear down and deploy it for $5,000, but in the past we would another, so hardware utilization is have had to purchase 15 to 20 desk- 7 SOFTWARE TESTING MARCH 2009
  6. 6. top workstations at $30,000 a piece. cloud computing. There are going to Now we can serve the same number be some changes to processes, and of employees with a lot less hard- you have to embrace that and look at ware.” the upside of the cost savings versus having 10 servers next to my desk. We find a lot of VPs of IT say it’s a threat CLOUD COMPUTING: to [their] way of life, in terms of head- THE NEW HYPERVISOR? count, budget, etc. We try to do as This type of capital expense will be one of the effects of cloud computing, according to Ravi Gururaj, CTO of The cloud will virtualization provider VMLogix Inc. enable QA to extend a in Palo Alto, Calif. “You’ll go from a its internal resources EDITOR’S LETTER CapEx to an OpEx,” he said, as organ- izations are billed for usage. “The to provision from cloud is really the new hypervisor. the cloud the amount a Instead of an ESX [VMware hypervi- of infrastructure CHAPTER 1 VIRTUAL sor] box, you will have a cloud box. required. TEST LABS MAKE MOST The cloud will give you the ability to OF CLOUD provision infrastructure on demand, but the tools themselves will be the much as we can to make it feel like a same, so the tools you’re using on- lab in your data center, with better a premise will be available in the cloud.” collaboration.” CHAPTER 2 TESTING RIAs Another benefit, he said, is that IDC’s Ballou said organizations also FOR SECURITY HOLES “the cloud can scale when you have need to examine how the vendor has a bursting situation.” For instance, structured its cloud strategy. First and in the final stages of QA, scalability foremost, she said, organizations may need to be tested. The cloud need to evaluate the stability and reli- will enable QA to extend its internal ability of the vendor and the service. resources to provision from the cloud For example, she said, is there a plan the amount of infrastructure required. for disaster recovery? Secondly, VMLogix is in the process of cloud- organizations need to look at how the enabling its products, Gururaj said. pricing model is structured and deter- The first stage will enable customers mine whether it is cost-effective: “Do that have VMLogix on premise to I have the resources to run this inter- burst to the cloud when necessary, nally, versus do I have the time? You and that will be followed by a pure have to balance that.” I cloud-based offering, he said. For organizations considering a cloud-based solution, Skytap’s Roza COLLEEN FRYE is a freelance writer and editor in Bridgewater, Mass. said it will require a mindshift. “We’re She has been covering the IT industry in the early adopter phase around for more than 20 years. 8 SOFTWARE TESTING MARCH 2009