Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Leading Tools And Solutions For Software Quality Assurance


Published on

  • Be the first to comment

  • Be the first to like this

Leading Tools And Solutions For Software Quality Assurance

  1. 1. Leading Tools And Solutions For Software Quality Assurance A SUPPLEMENT TO OCTOBER 1, 2006
  2. 2. t a b l e o f c o n t e n t s 4 introduction CEO PERSPECTIVE Rethinking Our Approach to Software Quality 6 ADAM KOLAWA, CO-FOUNDER AND CEO, PARASOFT Art? DIAMOND SPONSOR Parasoft Catches Bugs Before They Hatch—With Automation Science? 11 PARASOFT TEST SOLUTIONS LEADER Both! 15 IBM APPLICATION LIFE CYCLE MANAGEMENT TechExcel Takes the Guesswork Out of Quality Assurance 17 TECHEXCEL T esting is essential in every software development project. Yet while computer e ic h ic A la n Z to r, S D T im e s k DEFECT TRACKING D ir e c Axosoft Keeps Development Projects on the Fast Track E d it o ri a l 19 science programs teach devel- AXOSOFT opers about software architec- ture, object-oriented design, SOFTWARE CONFIGURATION MANAGEMENT algorithms and programming, body of knowledge has evolved Perforce Makes SCM Fast, Easy and First-Class 21 they offer little formal empha- best practices for functional PERFORCE sis on testing and quality testing, unit testing and per- assurance. formance testing, but now WEB SECURITY Keeping the Bad Guys at Bay With Cenzic Solutions Every day, managers im- those classic techniques are 23 CENZIC plore their programmers to being augmented by recent code faster. But do they pro- innovations in the field of TESTING SERVICES vide them with the tools and defect management, test Stelligent Brings Objectivity to Quality Measurement knowledge they need to write automation, software config- STELLIGENT better code? Do they equip uration management, metrics, their test teams with the best test design and security/vul- Thanks to Our Sponsors: resources to stamp out defects nerability testing. Diamond and vulnerabilities? All too Great software requires often, testing is neglected at great tools and great service every stage. providers. “The Art & Science Testing is an art. A develop- of Software Testing” profiles Platinum er or tester needs to understand leading test/QA solutions to where quality assurance fits help you choose the right part- into the application develop- ners for your projects. These ment life cycle, and why there’s companies can help your more to quality than passing a development and test teams Gold test suite or meeting some arbi- make better software. trary metric. We hope you enjoy this Testing is a science. Over special test/QA supplement to the past 30 years, a significant SD Times. & The Art& Science Editorial Director Lead Writer Alan Zeichick George Walsh BZ Media LLC Of Software Testing Managing Editor 7 High Street, Suite 407 Huntington, NY 11743 Customer Service/SD Times Subscriptions +1-631-421-4158 • fax +1-631-421-4130 Patricia Sarica +1-847-763-9692 • A SUPPLEMENT TO SD TIMES OCTOBER 1, 2006 President Art Director Ted Bahr Copyright © 2006 BZ Media LLC, All Rights Reserved LuAnn T. Palazzo Article Reprints Executive Vice President Lisa Abelson Alan Zeichick +1-516-379-7097 Copy Editor SUBSCRIBE TODAY! Laurie O’Connell Cover Photograph by Elena Korenbaum 1 october 2006 the art & science of software testing 3
  3. 3. c e o p e r s p e c t i v e Rethinking Our Approach To Software Quality T he primary mission of information technology is to tant lesson. Many people think that error prevention is not increase profits through improved business processes. possible in the software industry; they believe that because Companies are constantly rethinking and struggling each piece of software is different, the lessons learned from with how to use IT to a competitive advantage, reduce IT oper- working on one piece of software cannot be applied to oth- ating and maintenance costs, and reduce the total cost of own- er pieces. Instead of trying to prevent errors from entering ership…all while attempting to deliver increased value. software, the industry tries to test errors out at the end of Most of these challenges are directly linked to making the development life cycle. software work—without incurring unreasonable costs. Many First we build a product, then we test at the end of the people in the industry would agree that low IT productivi- production cycle to determine whether the product works, ty is the reason software development is so costly. But why and finally we remove any errors that testing exposes. are IT teams, with all their expertise and hard work, suf- Throughout this process, we cross our fingers and hope that fering from low productivity? The root cause of low produc- the most insidious and embarrassing problems will be iden- tivity is errors made throughout the software development tified before the release. However, a consideration of the life cycle. staggering number and impact of software errors reported These errors include everything from performance errors annually and their cost to the U.S. economy suggests that to security errors, to misimplemented functionality, to errors this quality-through-end-of-cycle-testing approach is not that crash an entire system. They essentially stifle IT teams’ yielding the desired results. ability to produce working software in a reasonable time and The belief that our traditional software testing approach at reasonable costs. In fact, if you look at virtually any IT can create quality software is a fundamental problem. We team, you will see that its members spend about 80 percent don’t think of the whole process of building and deploying of their time chasing and fixing bugs, and only about 20 software in a way that would prevent errors because we don’t percent of their time on tasks that deliver value and improve believe that it can actually be done. Yet, this error preven- the business. This practice is far from productive. tion approach is not only possible, it is necessary. The increas- Adding to this inefficiency is the traditional software development approach of leaving testing till late in the devel- opment life cycle. It is only then that QA does the testing necessary to ensure that bugs are found, requirements met, Testing, in general, must and reliability, performance and security goals achieved before an application is deployed into production. Finding and fixing errors late in the development cycle is exponen- become the responsibility tially more costly, time-consuming and inefficient than addressing them early and throughout the cycle. Maintaining this approach is a primary reason that we continue to strug- of every team member. gle with quality and low productivity in the software world. Many other industries have struggled with low quality, high costs and low productivity as a result of human error. The automotive industry, for example, recognized that although ing complexity of software systems, the push for faster, near mistakes cannot be entirely eliminated, they can be controlled. continuous release cycles, and the expanding dependency Those automotive manufacturers who, by taking a holistic on software for nearly every phase of business execution and preventative approach to the problem, making quality an require that error prevention be addressed. integrated focus throughout the production process and even If the software industry is serious about reducing the modifying their production lines to prevent as many errors error rate and resolving the issues that stem from errors, we as possible from ever entering the products, addressed their can’t afford to continue hoping that our current approach most critical problems and have remained viable. to testing will miraculously start yielding quality software. The software industry still has not learned this impor- Instead we need to follow in the footsteps of other indus- 4 the art & science of software testing 1 october 2006
  4. 4. tries and start preventing errors throughout the software development life cycle. Achieving Software Quality Achieving a consistently high level of quality starts with com- mitting to a practice of an end-to-end quality process. While there is no single silver bullet for producing reliable, high- quality software, there are proven steps that software devel- opment organizations can and should employ to help prevent software errors and improve development productivity. The most significant of these are addressed below. Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places a shared focus and importance on quality. Many companies con- tinue to treat development and testing as independent disci- plines. This separation of development and QA leads to many software problems and inefficiencies—developers might write code assuming that someone in the QA department will inject quality into the software. When problems occur, the code just bounces back and forth between departments without anyone taking responsibility. The manufacturing world learned long ago that it could not separate the responsibility of production from the Adam Kolawa responsibility of verification and expect to achieve quality. Co-Founder and CEO, Parasoft These responsibilities go hand in hand. Organizations should place development and QA under the same management or merge the two groups completely to facilitate owner- ship and responsibility for code quality. In an established group culture, developers will show that they care about quality practices. the code because caring about the code is synonymous with caring about the group. Automate Effective adoption of software quality practices Adopt Quality Practices requires automation of as many testing tasks as possible. Organizations must adopt software error preven- Development organizations are increasingly burdened to tion practices from the earliest stages of devel- produce more code faster and with the same or fewer opment. Well-known software error prevention resources. If they are to adopt software quality practices, practices such as coding standards, unit test- they will need to use software testing tools and development ing and regression testing are not regularly methodologies that allow them to automate many of these followed despite their recognized value in catch- practices and integrate them easily into their development ing errors at the code level early in the cycle when it processes. Fortunately, there are a growing number of prod- is easiest and least costly to find and fix them. These prac- uct offerings that deliver this automation and integration tices should be an integrated part of an organization’s devel- and that allow developers to take a “test-as-you-code” opment process. Testing, in general, must become the respon- approach to development—to readily create reusable test sibility of every team member. objects, and operate within a framework that facilitates cre- The trend toward service enablement of applications ating high-quality software. (e.g., Web services, SOA) and faster, shorter software release If their companies are to remain competitive, software cycles mandates that quality assurance and testing can no organizations must improve their productivity by control- longer be treated as a set event handled as an independ- ling their costs while ensuring the high quality of their deliv- ent discipline and relegated to a single proscribed phase erables. To do so requires rethinking and restructuring the of the development life cycle. It must become a continu- way we have traditionally approached software development ous, integrated part of the development process enabled and delivery and necessitates a heightened commitment to through the application of known software development quality throughout the software development life cycle and 1 october 2006 the art & science of software testing 5
  5. 5. d i a m o n d s p o n s o r Parasoft Catches Bugs Before They Hatch—With Automation E veryone knows that bugs exist. But at what stage of the development cycle should they be isolated and killed? Parasoft says, “Test early and often, when bugs are easiest plex, constantly changing enterprise systems like Java EE, SOA and Web services, reducing the risks of system down- time and security vulnerabilities. At the same time, teams and least costly to find and fix.” Parasoft’s Jtest software for can find more defects with their existing resources, increas- Java developers uses automation to make testing fast, easy ing productivity while adhering to budget parameters. and practical for developers One of the most exciting new features of Jtest is its to perform during the de- BugDetective. By automatically tracing and simulating exe- velopment life cycle’s cod- cution paths, BugDetective exposes runtime defects that ing stages, when testing would be difficult or even impossible to find through man- is usually the furthest ual testing or inspections. With BugDetective, you can now thing from a program- find, diagnose and fix classes of software errors that routine- mer’s mind. ly evade standard analysis and unit testing techniques. “If you look at a Jtest also lets development teams automatically generate developer’s resume, the and run tests using the popular Apache Cactus test frame- last thing you’ll see list- work. This gives organizations early, development-level defect ed is testing, because exposure that might go unnoticed until QA, deployment or developers just don’t production time, when it’s a lot more expensive and prohib- like to test,” says Brian itive to find and fix problems. Another new technology is Hunt, Parasoft’s VP Jtest Tracer, which creates realistic, functional JUnit test cas- of sales and acting es that reflect an application’s correct functional behavior. COO (www.parasoft With Jtest Tracer, organizations can quickly create libraries .com). “However, at of regression test cases that can be run to ensure that new some point you code changes don’t inadvertently break existing application have to prove that functionality. Hunt the software works. “The key to reducing testing time is automation,” Hunt B r ia n g COO a n d A c ti n We help developers declares. “Jtest can even perform testing overnight to scan VP o f S a le s validate what they’ve the code, find errors and report those errors to the devel- built from the point of cre- opers when they start working in the morning. It lets them ation to the completion of development. That drill straight through the results to the lines of code that validation starts at the desktop in the same way that spell need to be fixed. It can also perform automatic functional checkers are run against text documents.” tests that run the code to make sure that it does what it’s Jtest provides Java development teams an automated unit supposed to do. Because we write these tests in an open testing and code analysis tool suite that performs compre- format, you can modify and extend them to meet your spe- hensive test and analysis of Java source code, exposing bugs cific needs.” and errors in code structure, execution and design at the Jtest integrates with com- source or unit level. Used as a plug-in to the developer’s IDE plementary Parasoft prod- (such as Eclipse, WSAD, Rational RAD) or integrated with ucts to provide automated a central build process, Parasoft Jtest is designed to be used systemwide testing solutions by development teams in a “test-as-you-code” strategy to for Web applications, Web services and other n-tier systems. find and eliminate errors early in the development process Moreover, Jtest works as part of a comprehensive teamwide before they can infect the main application codebase. Automated Error Prevention solution that provides central- “Uncovering errors early and at their source or root cause ized administration and application of test practices, man- makes them quicker and less costly to fix, and helps reveal agement dashboards and metrics for real-time analysis that and resolve design errors that could have extended negative help managers evaluate code compliance, code readiness and impact on an application if gone undetected,” states Hunt. team productivity. The newly released Jtest 8.0 adds new testing innovations With Jtest you catch bugs before they hatch—early and to help teams automatically verify the functionality of com- often. & 6 the art & science of software testing 1 october 2006
  10. 10. a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t TechExcel Takes the Guesswork Out of Quality Assurance ning process. The test team receives its test assignment in the W hen it comes to software testing, two key words are speed and accuracy—not only of the application you’re test- ing, but also of the QA process itself. That’s what teams need— DevTest interface, executes the items assigned to it, and sub- mits defects directly from the interface into an integrated defect- and what TechExcel delivers in DevTest, one of the three tools tracking tool. Meanwhile, DevTest tracks the test results in a in its DevSuite. DevTest offers your developers test standardi- real-time dashboard and in presentation-quality custom reports. zation, test reuse and powerful defect analysis capabilities, while In addition to DevTest, TechExcel ( offers giving managers a bird’s-eye view of the entire process. two other products in the DevSuite: DevTrack and DevPlan. “DevTest is an integrated solution that allows QA teams to DevTrack tracks and improve standardization and leverage existing testing knowl- manages product defects, edge to carefully monitor test execution,” says Tieren Zhou, Tech change requests and oth- Excel’s CEO and chief software architect. “DevTest accomplish- er issues, facilitating team- es that goal by focusing knowledge management, test library work among users, teams creation, planning and scheduling, and test execution and analy- and customers. DevTrack also provides workflow and process sis. I think our approach is truly unique to the market.” automation, robust searching and reporting, and point-and- DevTest is a one-stop information resource for your team, click customization. storing all related test documentation, including requirements The newest member of the DevSuite, DevPlan, is an inno- documents, specifications, automation scripts, screen shots vative project-tracking tool designed exclusively for application and other essential components, in a central repository. This life-cycle management. DevPlan unites project tracking and gold sponsor “knowledge view” can be issue management, and incorporates configurable workflows, used in test creation and notifications, meeting requests and process automation. execution so that the test Zhou explains that in order to be effective, a QA organiza- team is always equipped tion needs both preparation and education, and that TechExcel’s to plan and execute its tools can help. assignments. “DevTest provides all of the materials needed to create and A test library is built on execute test assignments in one location so that leads and man- this knowledge founda- agers have the information they need to craft effective tests, tion, letting your team and testers have the information they need to execute them,” reuse tests for new test he says. “DevTest’s planning wizards are a favorite among assignments, new soft- test managers because they allow them to query existing data ware versions or even to help plan their assignments and reduce guesswork. In addi- entirely different soft- tion, the built-in integration with DevTrack allows testers to ware products. These execute their test assignments and log and regress defects from standard test pro- a single interface.” cedures, called test In the near future, the DevSuite will support distributed templates, can be deployment, allowing global development organizations to linked to supporting achieve the benefits of local performance even when working documentation in the on a global project—stay tuned for more about this exciting knowledge view. Test development. Zhou ect T ie r e n wa r e A r c h it templates can be or- DevTest can help both small and large organizations get a C h ie f S o ft CEO and ganized and classified handle on quality assurance. “DevTest is useful in any develop- based on products, applicable ment environment,” Zhou says. “It’s well suited for large glob- environments, functional areas or any other al development organizations because it provides them with a structure on which a team needs to focus. scalable, real-time view of their test projects regardless of whether Once a test library has been created, QA managers and team the tests are executed by their core team, by an outsourced team leads use a wizard-driven interface to assign tasks to test teams, or by auto-testing tools. However, even smaller development helping them leverage everything in the test library, including teams benefit from the ability to create, manage, analyze and previous test assignments and defect history, to aid in the plan- reuse their test coverage.” & 1 october 2006 the art & science of software testing 15
  11. 11. Ship Software OnTime. ™ The Fast & Scalable Team Solution for... Defect & Issue Tracking • Feature & Change Tracking • Task & To-do List Tracking • Helpdesk Ticket Tracking OnTime is the market-leading project, defect and feature management tool for agile software development and test teams. OnTime facilitates tracking, analyzing and trending team-based software development efforts in an intuitive and powerful user interface. A fully customizable UI, powerful workflow, process enforcements, two-way email communications and custom reports combine to help software development teams ship software on-time! Available for Windows, Web & VS.NET 2003/2005 OnTime 2006 Professional Edition OnTime 2006 Small Team Edition • For Teams up to 10 Members • For Teams of 10 to 1,000 Members • Free Single-User Installations • From $149 Per User • $495 for 5-Team Members • $995 for 10-Team Members 800·653·0024 software for software development™ Only $495 for up to 5 Users • Only $995 for up to 10 Users Free Single-User Installations
  12. 12. d e f e c t t r a c k i n g Axosoft Keeps Development Projects on the Fast Track M anaging the entire scope of one or more software devel- opment projects can be daunting. Within the sea of tasks, features, defects and test cycles among team mem- important items from slipping through the cracks,” he explains. “Instead of wasting developer time with trivial updates and report requests, managers can easily pull the bers, managers, stakeholders and customers, just figuring information themselves. OnTime 2006 frees developers from out who’s working on what can be a challenge. That’s where most, if not all, of the process-related overhead that sur- Axosoft’s OnTime 2006 rounds a project, and it helps them focus on what they do gold sponsor enters the picture. best: building great software.” OnTime 2006 is a Developers who use OnTime 2006 see a clear, intuitive defect management sys- view of all of their projects, including all the issues, defects, tem designed to help soft- feature requests, milestones and tasks. As they complete ware development teams tasks, developers escalate those items to subsequent work- ship software on time. flow steps. This can be set to automatically e-mail other With a focus on ship- team members when the next set of actions is ready to be ping software on time, performed. it not only effectively OnTime 2006 allows project managers to define speci- tracks and manages fications, workflows and security rules. They can create “what bugs, it effectively if” scenarios that generate predictions for completion times, addresses the broad- workload distribution and other milestones. During the pro- er challenges and ject’s execution, project managers have access to total proj- best practices of ect visibility: They know who is working on what, the progress project manage- being made on tasks, where bottlenecks are occurring, defect ment. rates and estimated completion times. OnTime 2006 After a project has been completed, managers can take offers maximum advantage of the accurate project history that continues to a flexibility for adminis- reside within OnTime 2006. This information can be used uceav Dan S h it e c t trators and users alike, for anything from measuring various productivity rates to ware Arc C h ie f S o ft with ready access from providing a basis for decisions concerning future projects. a Windows client, a Web Thus, past projects become part of a living knowledge base browser, or within Visual Studio. The highly that can be consulted at any time. configurable and customizable application can also be inte- Axosoft even offers a Customer Portal add-on for OnTime grated with leading SCM packages, including Perforce SCM, 2006 that embraces customers as participants in the process SourceGear Vault, Subversion and Visual SourceSafe. of shipping software on time. It provides a Web interface In addition to tracking defects, features and tasks, the tied to the OnTime database where customers can submit software governs projects with highly customizable workflow bugs and other input. and security rules. It also offers e-mail notifications and con- Customizable security set- versation thread tracking, time tracking and work logging, tings determine how much custom fields and reporting, audit trails and archiving. project information will be While many of Axosoft’s clients switch to OnTime 2006 visible to customers. This from other products, utilizing its importing functionality, functionality is especially useful for consultants and ISVs others seek to upgrade from a manual process. during beta-testing phases. “Traditionally, development tasks, requests, defects and More functionality is in the works for the product. other items that occur over the course of the development “OnTime 2006 is already a tool designed to enable the entire cycle have been tracked in spreadsheets, providing ample development team to ship software on time,” explains opportunity for human error and miscommunication Suceava. “While today it meets the needs of project man- between teams,” says Dan Suceava, chief software architect agers, developers and testers extremely well, future versions for Axosoft ( will provide further functionality for support professionals, “OnTime 2006 keeps track of everything and prevents IT directors and executives.” & 1 october 2006 the art & science of software testing 17
  13. 13. s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t Perforce Makes SCM Fast, Easy and First-Class S oftware configuration management isn’t a luxury—for any modern software development team, it’s a necessi- ty. But when an SCM system is as easy to use, easy to admin- of image files that can be viewed from within the applica- tion’s cross-platform graphical client, P4V. Regardless of whether you’re working with text or binary artifacts, Perforce ister, and full of productivity-enhancing features and bene- handles the job with style. fits as Perforce’s system, it’ll feel like all of your developers Perforce’s client/server are traveling in first class, all the way. application operates over “The Perforce SCM System lets teams of local and dis- any network or the In- tributed developers share project files that are centrally stored ternet, and includes its and managed by the Perforce Server,” says John Walker, own internal database Perforce’s principal product consultant. “The Perforce Server engine—saving you money, improving performance, and get- handles user requests and tracks all development activity in ting the system up and running fast. Installing the Perforce the built-in Perforce database. Each file’s state information Server is simple: Download the installation set from the com- can be quickly discerned from any of Perforce’s cross- pany’s Web site and run it. Installing any one of a variety platform clients. This means that a user can see when a ver- of Perforce clients is also straightforward, and the resources sion of a particular file has been updated, deleted or added needed to manage the installation are minimal. Even large to the server. Icons associated with the files indicate whether distributed sites of 2,000 or more users can be maintained they’re currently being edited, added or deleted by other by a single administrator, explains Walker. developers.” While Perforce has the basics down pat, the real payoff is gold sponsor The Perforce SCM System in the extras that the software’s creators developed to make ( lets your own coders more efficient. Perforce’s intelligent branch- each developer on your team ing mechanism lets developers work on different release obtain and refresh a local, branches of a particular application in parallel. Code lines private copy of versioned supporting specific releases are clearly and visually defined files by synchronizing them in the system. Once a branch is created, the relationship it with files stored in a file shares with the parent branch is tracked in the server’s meta- depot. Since all of the data. Since the ancestry of files is tracked between branch- metadata is centralized, es, the integration history is maintained in the metadata. file state information can That makes it easy to see which changes have and haven’t be gathered quickly and been integrated between the related branches—whether you’re easily. Perforce efficient- working in Java, J2EE, .NET, C/C++, C#, Visual Basic or ly manages both bina- HTML. ry and text files, from Perforce provides a basic defect-tracking system called jobs. source code to graph- A job typically represents an enhancement request or a bug ics to documentation. to be fixed. Job definitions are customizable to support work- That’s one reason flow, and jobs can also work with leading third-party defect why Perforce has tracking systems. Support for Mercury’s Quality Center is been adopted not only planned for the next release. This integration will allow users r by pure development to enter bugs in Quality Center and have them replicated W a lk e John n s u lt a n t Pr oduct Co shops, but also by into Perforce as jobs. P ri n c ip a l chip and hardware “The Perforce SCM System’s high performance is the prod- manufacturers that maintain uct of a streamlined architecture and closely integrated imple- large binary assets. mentation—not expensive server or network hardware,” For example, Walker says the Perforce System is very pop- Walker says. “The Perforce Server does not require dedicat- ular in the game development industry, where artists are ed hardware, and client workstations never need upgrading. required to create and store large numbers of large image The system’s networking capabilities aren’t a significant load files. To help all users—not just software developers—the for a typical LAN. With little need for customization or con- Perforce software automatically renders thumbnail versions figuration, you can be up and running in minutes.” & 1 october 2006 the art & science of software testing 19
  14. 14. w e b s e c u r i t y Keeping the Bad Guys At Bay With Cenzic Solutions Cenzic’s own security experts—and that means that your devel- T he faster you find a security problem in the application life cycle, the better. If developers catch vulnerabilities early, they can be fixed prior to pushing apps off to QA and customers. opment team can build secure software even if it doesn’t have expertise in that area. ClickToSecure accesses applications Fixing problems after they slip into a product can cost far more using a combination of Hailstorm technology and the Cenzic than catching them early. And of course, if you don’t find the Intelligent Analysis (CIA) Research Lab to run assessments. defects, someone else will. In fact, that CIA Lab is the foundation of the SmartAttack gold sponsor “Cenzic solutions provide Objects Library. Cenzic provides continuous updates based on tremendous efficiencies and new vulnerabilities through the lab, similar to an anti-virus an immediate return model to help you stay ahead of the exploits and attacks. You on investment,” says John can use the library just as it is, plus you can use Cenzic’s Weinschenk, Cenzic’s SmartAttack Objects as templates that your own security archi- president and CEO (www tects can customize for your organization’s special require- “By auto- ments, if necessary. mating the security test- In addition to its software applications and services, Cenzic ing process, our cus- offers security training courses that give customers the knowl- tomers can secure their edge and skills to use and maintain its products successfully. applications faster and The company also offers consulting services that include on- less expensively. Like site assessment methodology, and software-engineering and performance and consulting implementation with professional vulnerability functionality testing, consultants who have experience in penetration testing and security testing ethical hacking. should be automat- Among Cenzic’s many customers are Boston College, Debt ed. Vulnerabilities Exchange, IRIS Link and K2 Networks. Boston College’s devel- are an open invita- opment group uses Hailstorm in-house throughout its infor- che nk tion to hackers. All mation security group to test all university Web applications. W e in s u ti v e O ff ic er John h ie f E x e c Web applications and As they find security weaknesses, developers use Hailstorm P r e s id e n t and C infrastructure can be reports to remediate those vulnerabilities. The same group also tested with Cenzic solutions addresses its regulatory compliance issues with Hailstorm. to locate security problems.” In the case of Debt Exchange, IRIS Link and K2 Networks, Cenzic’s two key security products are Cenzic Hailstorm and ClickToSecure is put to work to perform security assessments. Cenzic ClickToSecure. These solutions provide the most com- Customers call or fax requests to test their applications, and prehensive and accurate tests in the industry. Weinschenk says Cenzic experts test them remotely using Hailstorm in collabo- that while most vendors use scanning technology to focus only ration with the CIA Lab’s expertise. Detailed results are then on commonly known vulnerabilities, Hailstorm uses Cenzic’s presented to the customer patent-pending Stateful Assessment technology to automate along with detailed remedi- penetration testing by using a series of transactions to identify ation information. vulnerabilities. Weinschenk says that Plus, Hailstorm’s SmartAttack Objects Library takes the Cenzic solutions are a perfect fit for development managers software beyond merely finding security holes—it can also help and other executives. Using a dashboard, customers can view enforce internal policies as well as bring organizations into applications in the testing phase, as well as the number and regulatory compliance with rules like the Gramm-Leach-Bliley types of vulnerabilities that are found. Act (GLBA), SB1386 and Sarbanes-Oxley, and the Payment Weinschenk adds that testing is extremely important Card Industry (PCI) Data Security Standard, and with best because susceptibility to intrusion can result in major recovery practices including SANS and OWASP, in addition to many costs and regulatory penalties. He also reports that Cenzic other regulatory standards. hasn't tested a single application that was not vulnerable. But ClickToSecure is a service that allows developers to make use don’t worry—Cenzic finds the vulnerabilities before the bad of Software as a Service (SaaS). Tests are conducted remotely by guys do. Put Cenzic to work for you. & 1 october 2006 the art & science of software testing 21
  15. 15. Don’t Miss Out On Another Issue of The Test & QA Report e-newsletter! Each FREE weekly issue includes original articles that interview top thought leaders in software testing and quality trends, best practices and test/QA methodologies. Get must-read articles that appear only in this e-newsletter! Sign up at:
  16. 16. t e s t i n g s e r v i c e s Stelligent Brings Objectivity To Quality Measurement M any tools exist that help QA departments and programmers test applications. However, it’s also crucial that senior- level software managers have the necessary information to inter- Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring, and introduced developer testing as an act with executive management and accurately assess the effec- active part of the continuous integration process. The customer tiveness of their development teams. “If addressing software qual- found that Stelligent’s Quality ity early in the development cycle is an important priority for your Roadmap quickly solved the company, Stelligent can help,” says Burke Cox, Stelligent’s CEO. problem, and then its Con- “As the thought leaders in early testing, we use commercial tinuous Quality program kept and open-source technologies to assess and manage software them solved. Case closed. quality during the development and assembly of applications,” “For many organizations, our Quality Risk Index is the first Cox says. “The Quality Risk Index provides an objective meas- objective measure of their software quality that they have ever ure of software quality that enables an entire organization to seen,” Cox says. “This helps the software manager by providing assess quality and progress.” an independent assessment of what their quality practices cur- Cox believes that Stelligent ( transforms rently achieve. When working with the executive management the way software is developed and tested. By introducing com- team in developing schedules, approving budgets and other prod- prehensive inspection as part of a continuous process, both the uct management activities, the software manager uses Stelligent development team and senior management gain crucial visibil- reports as a basis for discussion. For example, high code com- ity into the quality risks associated with software projects. This plexity with poor test coverage might make a compelling argu- silver sponsor real-time feedback lets organ- ment to delay release, as opposed to simply stating that the prod- izations manage quality long uct is just not ready for prime time.” before products are ever Stelligent’s services provide a critical and objective meas- delivered to the QA team for urement of your software’s quality. Use this intelligence to evaluation. make it difficult to enter new defects into the source repos- Stelligent’s services, such itory, helping your developers leverage the continuous qual- as its Kickstart Quality Risk ity feedback before code even reaches the QA department. Assessment, are used by Stelligent is a subsidiary of JNetDirect, which makes organizations developing Convergence, a solution for providing real-time visibility into software using managed software quality; CoView, which helps your team develop accu- languages, like C# or rate and effective JUnit tests; and also high-performance JDBC Java, providing both drivers. But long before JNetDirect acquired Stelligent, it was a static and dynamic customer, using the Kickstart Quality Risk Assessment services analysis on the appli- to help build its own products. Stelligent’s impact was so dra- cation source and matic that JNetDirect immediately saw the value of bringing compiled code. Stelligent’s services to a broader audience. How effective is “Defects will always enter the build system,” Cox says. “Our the Kickstart Quality business is ensuring that they cannot live there for very long. Risk Assessment? Well, Quality may not have a material impact on initial product rev- C ox one customer recently enue, but it is the most significant driver of product profitabil- Burke e O ff ic e r acquired a software ity. Most companies struggle with describing quality; it is mixed C h ie f E x e c u ti v product that it believed with subjective measures and anecdotal evidence. Our customers was of high quality, based on a have a quality score they can point to when setting goals and due-diligence assessment of the source code and benchmarks. For organizations interested in increasing earnings, end user experience. However, when trying to integrate the prod- lowering the costs associated with poor product quality is the uct into its larger solution, it found the product had poor toler- best place to start.” ance to change. Stelligent’s services start with the Kickstart Quality Risk That’s when the customer called Stelligent, whose Kickstart Assessment—ask the company how its objective measures can Quality Risk Assessment identified the problems. Then, its help you, too, improve your software quality. & 1 october 2006 the art & science of software testing 23