Smarter Quality Management: The Fast Track to Competitive Advantage Organizations that create and deliver customers, even a massive recall or the software—whether fortheir own IT complete failure of a system at a critical operations, for the packaged applications moment. Get these things right, and you market,or as the core of their final product, can achieve a positive operational return as in the systems space—must grapple not on investment from efficiencies gained in only with today’s tough economic climate, development activities. but also with increased complexity in their processes and supply chains. Many factors One of the biggest challenges related to serve to complicate software delivery, quality management is how to invest but competition lies at the heart of this intelligently to minimize risk, given economic complexity. constraints. However, figuring out a) how to relate quality to business outcome and b) Here are a few examples. In the products what constitutes the right level of quality for arena, customers demand more from the individual products is not always clear. software components designed for thelatest hardware, often with requirements that This paper introduces a practical approach change rapidly, even as software projects to quality management (QM) that helps are underway. Keeping track of changes reduce time to market without sacrificing while meeting aggressive (and unaltered!) quality in the outcome. The underlying deadlines is difficult, ifnot impossible. In concepts presented here will be familiar1 the IT space, more businesses are focused to software project managers, especially ontheir operational software for capturing those with QM experience, but I will explainPAGE and providing value to their customers and some fundamentals as we go along to lines of businesses. E-commerce websites ensure all readers seeking these benefits compete to improve customer relations can understand the essential processes and simplify online business; businesses involved. that create highly optimized supply chains supporting a fast, efficient ecosystem of partners quickly rise in the market place. The nature of What does this competitive environment software mean for businesses seeking to deliver high-quality products and services? development Certainly, effective quality management creates opportunities to deliver key business Here’s one way to understand the “soft” in benefits, such as improved market share, software: it is relatively easy to change. But for higher customer satisfaction, and increased software designed for the commercial space, brand equity. But top quality in the completed where the competitive pressures described product cannot serve as the single guiding above govern a software project’s success principle by which products are produced or failure, the “soft”feature happens to be and delivered. Time to market is also key; one of its riskiest attributes. That’s because costs and risk factors must also be part of software projects are seldom designed and the balancing act. Get these things wrong, manufactured as in traditional engineering and you may face unsustainable costs, projects—a bridge, for example. While a missed windows of opportunity, unhappy bridge is engineered through traditional planning and architecture based on the laws
Smarter Quality Management: The Fast Track to Competitive Advantage of physics, then produced according to an organized plan with a division of labor, Quality software is, at its essence, simply information. Its development typically resembles a more management in creative process than one bound by the laws of nature. Walker Royce, IBM Software the software Group/Rational’s chief software economist, compares software production to movie development life production: a collaboration involving a team cycle of craftsmen and emerging from the naturally creative process of artistic yet technical What is the role of the testing, or QM, team people. during the iterative life cycle? What do they test for, and how do they know what is Over the past two decades, this unique supposed to change from one iteration to feature of software has been understood the next? and embraced by iterative development practitioners, who now tend to develop As noted above, traditional software testing software in stages called “iterations.” Each only occurs late in the life cycle, after iteration delivers a working, functional multiple coding teams have spent much version of the software under development, time and effort to deliver their components so it can be reviewed, tested, and toward the complete project. Because these2 vetted by stakeholders and other teams traditionally managed projects proceed seeking adherence to the original project according to strictly described requirementPAGE requirements. This allows project managers sets, and various component teams focus to make smaller, incremental course on their portions alone, it is up to the corrections during the project life cycle— testers to discover the discontinuities and thus ensuring the final deliverable is close malfunctions as these components are to expectations—as opposed to having assembled—then it’s the testers who must separate teams work according to a plan, deliver the bad news that much rework has assemble various components near project to be done inorder to get the project back end, and discover major failures due to on track. integration or deployment complexities. Iterative software development techniques For testing teams, the iterative development improve on that scenario by introducing process integrates quality management test teams to the process much sooner. A across the states of the project work flow, relatively modest, first iteration may only as opposed to relegating test activity to address 15 percentof the full set of project the end of the project. I will describe the requirements, but as a functional module role of iterative development-based quality of working code, the completed iteration management more fully in the next section. can be demonstrated, and tested. So any defects discovered by test teams at this early stage have a proportionally small impact on the larger development team, who make the fixes, then proceed to the next iteration where more of the requirements can be incorporated into the working version (iteration) of the software.
Smarter Quality Management: The Fast Track to Competitive Advantage The number of iterations required by any The best time to release varies widely software project depends on many factors, of based on your software delivery strategy course, such as the complexities of the team’s and your target market. Software delivery supply chain, the complexity of the software for both IT (internal business systems) and underdevelopment, the physical location smart products (products using embedded of team members (are they geographically software, including “system-of-systems” distributed, perhaps internationally? or are design) is typically dominated by one of two they co-located under a single roof?), and motivators, depending on the organization’s the competitive demands that determine target market: time to market (schedule optimum time to market. During any software driven), or quality impact (quality driven). delivery process, “When do we release?” is a key question withno simple answer. You Schedule-driven delivery implies “deliver must consider project-specific variables, on time, regardless of other factors” and such as the cost of delays, the opportunity is often used in industries where “Time to value of early delivery, marketplace quality market is king.” Consumer electronics is expectations, and the costs associated with one good example, as well as automotive, 4 Smarter qualityUltimately, track to competitive advantage defects. management: The fast the delivery strategy segments of the medical industry, and other will be based on the actual or perceived markets, where product teams try to gain a importance of each variable. first mover advantage over their competitors, (almost) regardless of the risk associated with inadequate quality. It should also be3 noted that schedule-driven delivery is not The optimal timePAGE limited to the systems space (i.e., the many to release embedded software devices industries). Many IT development teams use schedule- driven delivery when trying to enhance their The optimal time to release is when the total end typically experienceone of two motivators, depending The optimal time to release is user dominated by and increase market risk exposure isminimal, typically around share, taking away from their competitors, on the organization’s target market: time to market (schedule The optimal time to release is when the total risk exposure is the time where the risk associated with minimal, typically around the time where the risk associated driven), or quality impact (quality driven). often risking quality in the process. competitive threats starts to outweigh the with competitive threats starts to outweigh the risk reduction associated with further quality improvements, as illustrated in risk reduction associated with further quality Schedule-driven delivery implies “deliver on time, regardless Figure 1. Figure 2factors” and is often usedrisks associated to of other represents the in industries where “Time improvements, as illustrated in Figure 1. with schedule-driven software is one good example, as market is king.” Consumer electronics delivery. The green as automotive, segments of the medical industry, and other well line represents the risks associated with the where productof yourto product mover advantage markets, delivery teams try gain a ﬁrst being Risk Exposure Many High opportunity cost; over their competitors, (almost) regardless of the risk associated critical defects Strong competition reduced over time. It should also be noted that schedule- with inadequate quality. The red lines represent Lowest Overall Risk the risk delivery is not limitedstealing your market many driven of competitors to the systems space (i.e., the Exposure away increasing devices time, as Many IT development embedded software over industries). well as risks associated with opportunity when trying to enhance their teams use schedule-driven delivery costs. Few end user experience and increase market share, taking away from minor defects their competitors, often risking quality in the process. Low opportunity cost; Weak competition The intersection is the point in time where Time the sum2of both lines, i.e. the total risk, is the Figure represents the risks associated with schedule-driven Quality risk ( = Probability of defects x loss due to defects) lowest. As seenTheFigure 2,represents the moves software delivery. in green line this point risks associated with the delivery of your product being reduced over time. The Competition risk ( = Probability of competitors x size of loss to competition) to the left as the environment you are in is Total risk ( = Sum of all risks) red lines represent the risk of competitors stealing your market more competitive time, as well as risks associated with oppor- away increasing over in nature. (Notice that the intersection point is moving up as well.) tunity costs. Figure 1: Minimal risk exposure is when opportunity cost and competitive threats outweigh risk reduction related to quality improvements. The intersection is the point in time where the sum of both lines, i.e. the total risk, is the lowest. As seen in Figure 2, this The best time to release varies widely based on your software
Quality-driven delivery can also be costly but for different rea- Time to Market is king! sons. As shown in Figure 3, the more critical any defects might Smarter Quality Example: ConsumerThe Fast Track to Management: Electronics Competitive Advantage to get to an optimum be regarding quality, the longer it takes release point. Risk Quality-driven delivery can also be costly but for different rea- Exposure Time to Market is king! sons. As shown in Figure 3, the more critical any defects might Quality is king! Example: Consumer Electronics opportunity cost; be regarding quality, the longer it takes to get to an optimum High Example: Safety Critical applications Strong competition release point. Risk Risk Exposure Exposure Criticality of Quality is king! Defects High opportunity cost; Example: Safety Critical applications Strong competition Risk Exposure Criticality of Time Defects Figure 2: The blue dots show possible release points, with points of minimal risk moving forward as competition intensiﬁes (red lines). Time Time Quality issues are often magniﬁed in a schedule-driven life cycle, Figure 3: The more critical the implications of defects are, the more time it given that software contractors frequently get paid on a time and takes to get to the lowest risk point where release is possible. Figure materials basis,show possible release points, with points of minimal 2: The blue dots regardless of the quality of software they deliver. In many cases, you may even end up paying extra for the deliv- risk moving forward as competition intensiﬁes (red lines). The release timing for this approach is governed by achieving Time ery team to ﬁx their own defects, so the potential costs of defects the right quality, moving the optimal time to release further to Quality issues are often magniﬁed in a And here’s an interesting sta- to the end user can add up quickly. Quality According to areCarnegie schedule-driven life cycle, a Figure theThe A target based on“defects fixed” might issues the often magnified in use. more critical the do you deﬁne that? are, thedefects is practi- 3: right—but how implications of defects Zero more time it given tistic: Mellon Software Engineering that software contractors frequently get paid on a time and takes to get to the lowest risk point where release is possible. no way to deter- cally impossible to achieve, given that there is schedule-driven of lifequality 80 softwareof thedeliver. materials basis, regardless the 60 - of percent they cost that Institute, “Data indicate that cycle, given of be more realistic—but it’s still impossibleto software development is in rework.”1 extra for the deliv- mine how many defects still exist in a piece of code or the software you may even end upfrequently get paid The know the for this approachremaining achieving in the In many cases, contractors paying release timing number of is governed by defects ery team to ﬁx their own defects, so the potential costs of defects on aend user can add up quickly. And here’s an interesting sta- to the product. time and materials basis, regardless the right quality, moving the optimal time to release further to of the quality of softwareSoftware Engineering In tistic: According to the Carnegie Mellon they deliver. the right—but how do you deﬁne that? Zero defects is practi-4 cally impossible to achieve, given that there is no way to deter- many cases, you may even end cost of Institute, “Data indicate that 60 - 80 percent of the up paying mineRisk-driven stilldelivery of code or the delivering how many defects exist in a piece implies software development is in rework.”PAGE 1 extra for the delivery team to fix their own your software when the risk is minimal. defects, so the potential costs of defects But in practice, we always need to release to the end user can add up quickly. And “early”—earlier than we can. Which typically here’s an interesting statistic: According to implies increasing the risk, right? At least the Carnegie Mellon Software Engineering this is a commonly held view, but is it always Institute, “Data indicate that 60 - 80 percent the case? of the cost of software development is in rework.” Within the risk-driven model, the optimal release time is when risks are sufficiently Quality-driven delivery can also be costly reduced (not completely eliminated) and but for different reasons. As shown in Figure time to market has not been wasted. In 3, the more critical any defects might be other words, some time is needed to reduce regarding quality, the longer it takes to get the most significant risks, but the company to an optimum release point. cannot afford to address every known risk because the opportunity to beat the The release timing for this approach is competition is fleeting. governed by achieving the right quality, moving the optimal time to release further to So the question is, how can we get to this the right—but how do you define that? Zero point sooner? How do we compress the defects is practically impossible to achieve, release date from the optimal intersection given that there is no way to determine how (shown as a blue circle in Figure 4) to a point many defects still exist in a piece of code or earlier in time? the probability of detecting those defects in
Smarter Quality Management: The Fast Track to Competitive Advantage We cannot simply cut the time requirement,e because as we move left on the green line, Understanding the risk goes higher. But what if we could compress the curve described by the green quality line—push it down, so to speak? Then we could not only deliver sooner but lower the management: It’s overall risk as well. The intersection point will move down and to the left. This improved more than simply scenario is shown in Figure 4. testingget based onill impossible If a faster reduction in risk is the goal, roduct. how do you achieve it? The answer is not through testing, which is focused simplyware whend to release on discovering defects. In traditional testing Optimallies increasing Time to Release practices, testing is considered a late stageew, but is it activity, squeezed between an often-late development hand off date and an immovable time is when Time ship date. Not only does this practice fail toinated) and Time to Market yield the benefits of incremental, iterativeds, some timethe company development techniques explained earlier; 5 the opportu- Figure 4: To deliver early, at an improved quality, reduce your risk at an earlier point in the life cycle. it also minimizes, or at best reduces, the amount of time spent on quality assurance, PAGEooner? How Risk-driven delivery offers a practical improvement over these and makes fixes all the more difficult unless intersection Risk-driven schedule driven vs. quality driven) practical two extremes (i.e. delivery offers a because it you’re willing to compromise the release more cost-effectively balances quality versus time-to-market ier in time? improvement over these two extremes (i.e. date. considerations. A risk-driven strategy is a reﬁnement of a schedule driven that optimizes risk exposure because quality-driven approach vs. quality driven) againstuse as we movehat if we could it more cost and time. development cost-effectively balances quality As noted earlier, iterative developmentpush it down, versus time-to-market considerations. A techniques greatly improve this situation byer but lower For the remainder of this discussion, we will assume that thel move down risk-driven strategybased on a risk-driven approach. a software delivery life cycle is is a refinement of having functional units tested incrementally,n Figure 4. quality-driven approach that optimizes risk We will explore how to bend the green curve shown in Figure 4 in stages, throughout the life cycle, rather exposure to the left, fordevelopment cost and downward and against reduced time to market without than leaving the testing phase until project compromising the risk proﬁle. time. completion. For the remainder of this discussion, we And quality management takes this will assume that the software delivery life improvement a step further2.. Quality cycle is based on a risk-driven approach. management, which is the implementation We will explore how to bend the green of best practices to proactively reduce risk curve shown in Figure 4 downward and to throughout the whole life cycle, is a risk the left, for reduced time to market without reduction mechanism in its own right. By compromising the risk profile. choosing quality management practices with the potential to deliver a positive ROI within a relative short amount of time, you can justify risk reduction measures from not only a quality stand point but also a financial standpoint.
Smarter Quality Management: The Fast Track to Competitive Advantage Approaching quality from a full life cycle earn their paychecks by testing chunks of perspective—as opposed to assuming that completed, compiled software code against testing activity will suffice—should not seem the requirements that code is designed to like such a radical idea. After all, testing a fulfill. Requirements are mapped to “test product is an engineering task just like cases”—sets of conditions by which a development: the requirements need to be tester can know whether or not a particular analyzed by the test architect, its testing requirement is met. Each requirement strategy has to be defined, the relevant test demands one or more test cases. Naturally, benches and test frameworks need to be some requirements are more critical than built (designed and implemented), etc. These others, so some test cases represent higher engineering development processes are very value to the overall project than others. similar to the ones followed during product development. In fact, product development The benefits of traceability—that is, tracing and quality management can be viewed as test cases back to the requirements— two parallel development threads emanating are clear. It allows the testing team to from the same requirements, with many measure the testing coverage vis-à-vis your synchronization points between the threads, requirements: How many of the requirements up to the point of the testing activity itself have been covered? Which requirements where a verdict is made based on meeting have not been tested? And how many of expectations or not (as expressed in the requirements that are associated with6 the test cases). This applies to both the test cases pass? And of course you would traditional development process as well as like these answers to be across subsequentPAGE agile methods (with variants such as test- builds rather than a single one. first development and test-driven design). The important point is that QM is a thread But this is not all. Traceability allows for that must run in parallel to development, test execution prioritization, saving money especially in agiledevelopment. and time in the process. As each iteration proceeds, the number of test cases that may Although a complete discussion of QM is be involved increases as more and more beyond the scope of this paper, we can requirements are met. Most test teams feel it show how QM reduces risk—and thus is necessary to step through the entire suite allows earlier software delivery without of test cases touched by the latest iteration compromising quality—by demonstrating in order to thoroughly assess quality. But is how one of the quality management best this truly necessary? practices can improve iterative testing within the software development life cycle. No. Re-executing all the test cases, either in the agile context or as a growing set of regression tests in a waterfall process, increases the time required to test; it requires Test cases: further efforts to maintain the test cases, and it becomes a drain on productivity and, Driving testing ultimately, adds to the project’s bottom from requirements line cost. To reduce the wasted effort and associated costs, this all-too-typical use of test cases can be replaced with a more Simply stated, software testing teams results-oriented approach to requirements
Smarter Quality Management: The Fast Track to Competitive Advantage coverage, one that maps results to testing link the appropriate test artifacts for each activities. The essential principle is this: requirement, it would take approximately prioritize testing according to risk. As 10 person-months to create the traceability changes occur to working code during the between requirements and test cases. You could potentially reduce this time to one life cycle, test teams may choose to test to two minutes per requirement—and a for the most critical requirements first, and total of just 10 to 20 person-days—using maybe later test for all critical requirements a dedicated quality management solution remaining (for example, in another iteration). with support for capturing traceability links In other instances, they may choose to run between requirements and test cases. At a only the subset of test cases that provide nominal rate of US$50 per hour, this singleses back to choose to run only the subset of test cases that provide them them highest coverage. This approachcoverage. This with the highest is illustrated in change corresponds to a potential saving of am to meas- with the s: How many approach is illustrated in Figure 5. Figure 5. around US$75,000. And needless to say, inquirements real life, the numbers are much bigger.ementsrse you would There are other best practices that contribute No. of Requirements ther than a High Requirements coverage to improved quality at a reduced cost. We Most critical Requirements are not going to cover them all in this paper,tion prioriti- All critical Requirements but here are two for you to consider. Low contributionach iterationolved • Improved collaboration between the Most test QA team and other stakeholders: From suite of Test Suites talking to customers, we learned that 7 thoroughly on average, a tester spends only about PAGE 60 percent of the time performing actual ile context Figure 5: After a change, re-executing all the tests is safe, but expensive and testing, test planning, or test reporting. often unrealistic. By focusing instead on test suites that are the most relevant process, The other 40 percent is related to to the iteration that is being tested, test teams make more efficient use of their er efforts to time and reduce redundancies along with high testing costs. activities that are collaborative in nature,productivity st. To reduce such as clearing up the requirements ypical use of Although cannot completely remove risk from a develop- Although you you cannot completely remove with domain experts or business ment program, you can measure it and manage it by taking ted approach risk from mitigation actions. This approach helps ensure the appropriate a development program, you analysts, or exchanging emails testing activi- can measure it and manage it by taking that you are optimally using your ﬁnite testing resources (your and phonecalls with members of the g according the appropriate mitigation actions. This people) to reduce risk as rapidly as possible in the development development team. This gets worse inhe life cycle, equirements cycle. By focusing testing effort on high-value test cases, from approach helps ensurethatcontributionoptimally you are to business distributed organizations. If you could either the point of view of coverage or track and manage the collaboration,nts remaining using your finite testing resourcesyou value, you essentially prioritize according to risk—and (your es, they may it will not only reduce your risk people)about test cases that pertain to noncriticalpossible worry less to reduce risk as rapidly as issues. associated with lack of communications in the development cycle. By focusing testing and misunderstandings, but also reduce effort on high-value test cases, from either the time for collaborative tasks by 20 - the point of view of coverage or contribution 50 percent. to business value, you essentially prioritize according to risk—and you worry less about • Automated reporting: Creating a report, test cases that pertain to noncritical issues. especially one that goes to high level management, requires data collection But does creating the traceability links from many sources, sometimes from require an expensive investment? No, and teams that are in different time zones, here is the basic math. Consider a hypo- and then formatting this data thetical medium-size project with some appropriately. If you could automate this 5,000 requirements and 10,000 test cases. activity, your team will probably use Assuming it takes 20 minutes to locate and it more often and take the appropriate
Together, these quality management best practices can beneﬁt and the Smarter Quality Management: The Fast Track to Competitive Advantage Using CMMI the overall business in measurable ways. 3 as a by appl proxy for the maturity of the development process, Figure 6 to 58 p shows that the overall business impact of quality management 1914 (5 decisions in “real time,”thus reducing is quite compelling. more defects. 3300), or 914 As ﬁxin your risk. How much would this save seven ti you? and ass Impact of Quality Management on Process Efficiency gration These are some of the quality management 100% 90% 85% 85% 87% cost by best practices, each of which contributes to 80% 76% 75% And th risk reduction and therefore increased quality 70% 60% 58% 60% numbe and reduced cost. Now, let’s consider the 50% the ﬁrs overall impact of quality management on the 40% 30% 32% 30% This al defects density and the cost of fixing them. 20% 15% as incre 10% of quali 0% 1 2 3 4 5 QM Impact: 20% 40% 40% 40% 10% As mos CMMI Levels W/O QM W QM The overall 2 or 3, most of Figure 6: Graphing percentage of defects detected (Y axis) against an mature business impact of organization’s software development maturity level (X axis). 4 and 5 more im quality management Let’s assume an organization at CMMI level 2, with 1000 defects of defec As fixing defects during User Acceptance detected during functional testing. Figure 6 shows that on improv Testing (UAT) is aboutseven times more average, without QM practices, about 30 percent of the defects related are being detected in functional testing (the left, blue bar), test, expensive than during unit/integration as a dif Quality management best practices center on quality synchronization points across and assuming a fix cost of US$120 per the whole development life cycle. We have defect during unit/integration test, fixing 914 discussed the benefits of traceability to defects in UAT is already increasing the cost8 requirements, and we have briefly noted by over half a million dollars! collaboration between stakeholders and thePAGE QA team, as well as automated reporting. And this does not even take into account Other best practices include: allowing quality the reduction in the number of defects that professionals to contribute to the team effort result from applying QM practices in the from the very beginning of a project; the first place, which makes the savings even integration of practitioners doing the testing more significant. This also does not take as part of quality management; and the use into account less tangible savings, such of consolidated quality dashboards. as increased quality, customer retention, and other implications of quality as a Together, these quality management best differentiating asset. practices can benefitthe overall business in measurable ways. Using CMMI3 as aproxy As most software development teams are for the maturity of the development process, around CMMI levels 2 or 3, the benefits and Figure 6 shows that the overall business the savings described above apply to most impact of quality management is quite of the industry. But as development teams compelling. become more mature, apply QM practices, and move up to CMMI levels 4 and 5, the focus Let’s assume an organization at CMMI level 2, shifts into less obvious—but for some, even with 1000 defectsdetected during functional more important—benefits, such as reduction testing. Figure 6 shows that on average, in the number of defects that are introduced without QM practices, about 30 percent of in the first place, measured improvements the defectsare being detected in functional around planning and execution of quality testing (the left, blue bar), and therefore the related activities, customer retention, and total number of defects is 3300. However, by leveraging quality as a differentiating asset. applying QM practices, the defect detection rate increases to 58 percent (the right, green bar), therefore detecting 1914 (58 percent of
Smarter Quality Management: The Fast Track to Competitive Advantage Better quality + For more lower cost information = improved To learn more about the IBM Rational quality management offerings, please contact competitiveness your IBM marketing representative or IBM Business Partner, or visit the following In this paper, I have described several website: improvements to methods used by software ibm.com/software/awdtools/rqm/ teams in the design, testing, and deployment of software for systems or IT. Teams may Additionally, financing solutions from IBM use these quality management methods to Global Financing can enable effective cash deploy that software more quickly, while management, protection from technology mitigating quality-related risks throughout obsolescence, improved total cost of the life cycle. The multidisciplined practice ownership and return on investment. Also, of quality management is breaking the our Global Asset Recovery Services help functional and organizational silos that address environmental concerns with new, are so common in today’s companies. It more energy-efficient solutions. For more encourages an analytical process that’s information on IBM Global Financing, visit: closely integrated with the development life ibm.com/financing9 cycle.PAGE Analyzing the market and best practices shows that business outcomes can be About the Author optimized, and that smart improvements within the realm of proven best practices for Moshe Cohen is the Market Manager for IBM requirements management and traceability, Rational quality management offerings. In his collaborative test planning and automated current role, he works closely with customers, reporting—a combination of disciplines including managers and practitioners, to that defines quality management—can help drive IBM Rational quality management address the need for increased innovation offerings in both the IT and embedded with more competitive products and services systems spaces. Prior to this, he was with to your customers. Telelogic, defining and driving its Model Driven Testing solutions. He has extensive The IBM® Rational® organization is ready hands-on experience in the specification, to demonstrate these techniques to you. development, and testing of C3I medical and With straightforward adjustments to your telecom applications, including technology investments, deployment practices, and adoptions and driving process improvement tooling, we can help you realize these programs. He received his EE and M.Sc in benefits within a time frame that best suits mathematics and computer sciences, both your business’s needs. with honors, from Beer-Sheva University in Israel. We look forward to working with you!
05 – 08 November 2012 | RAI AmsterdamThe EuroSTAR Conference attracts testers from around the globe whocome to learn, network & discuss all things testing. Attendees experienceintensive tutorials, track sessions & inspirational keynote speakers, & can visit Europe’s largest Software Testing Exhibition. Return to the office inspired, motivated, and full of new ideas! www.eurostarconferences.com
Join the EuroSTAR Community…Access the latest testing news! Our Community strives to provide testprofessionals with resources that prove beneficial to their day-to-day roles.These resources include catalogues of free on-demand webinars, ebooks, videos, presentations from past conferences and much more... Follow us on Twitter @esconfs Remember to use our hash tag #esconfs when tweeting about EuroSTAR 2012! Become a fan of EuroSTAR on Facebook Join our LinkedIn Group The EuroSTAR Blog Contribute to the EuroSTAR Blog Written by Testers for Testers Check out our free Webinar Archive Download our latest ebooks w w w. e u r o s t a r c o n f e r e n c e s . c o m