Cover story on STAG Software and HBT in The SmartTechie magazine.

1. ` 25 only
Guru talk: Sashi Reddi, AppLabs VC Talk: Jagannath Rao, Headstart Ventures
EMPOWERING TECHNOLOGY LEADERS OF TOMORROW October 2010 thesmarttechie.com
T Ashok, CEO
STAG Software

2. COVER STORY
STAG In
Pursuit
software
of Clean Software By Vimali Swamy
STAG Software at a glance
here has been significant make guarantees, it is imperative that
Founded: 2000
CEO: T Ashok
Headquarters: Bangalore
Other Offices: Chennai, San Jose
T progress in the field of
software testing in re-
cent years. However,
one finds it hard to pro-
vide a logical means to get closer to
perfection. A typical approach to test-
the approach to evaluation be sharply
goal-focused. It is to achieve this
quality of guaranteed clean software
that STAG Software is striving on.
Founded in 2000, STAG Software is
a boutique test engineering company
Employees: About 200
(U.S.), Tokyo (Japan)
ing based on the activity-based headquartered in Bangalore.
Customers: Over 100
model consists of strategizing, plan- By goal-focused evaluation, the
Website:www.stagsoftware.com
ning, designing, automating, execut- company means that one should
ing, and managing. Over the years, have clarity as to what potential de-
the industry has moved from com- fects we have to go after. Once the
pleting these activities in one go in to potential defects are discerned by
an agile version consisting of these employing a scientific approach, it
activities done in short increments is possible to arrive at an effective
and the field of software testing has validation strategy, a complete set
been littered with jargons, process of test cases, better measures of
models, and tools. Yet the notion of cleanliness (quality), and appropri-
‘guarantee’ seems elusive. ate tooling.
Is it possible to guarantee qual- Having pioneered a scientific
ity? ‘Guarantee’, implies that the de- approach to testing – Hypothesis
ployed software will not cause based testing (HBT), it is attempt-
T Ashok
business loss and that the means of ing to swim against the current by
validation can be proven to be cor- defying the traditional approach to
rect. It is generally understood that testing. Today, a decade since its
testing is a process of uncovering de- humble beginnings, HBT is being
fects done via a good mix of tech- slowly accepted as a more result-
niques, tools, and people skills. To oriented option by the industry.
The Smart Techie
|33| October 2010

3. HBT – Hypothesis Based Testing tential problems based on one’s symp- was that most of the test cases were gradeability and maintainability. by a jaw-dropping 10x times. These potential defects are hypothe-
Hypothesis Based Testing (HBT) is a toms, performs diagnostic tests to con- not detecting defects and the rate of In another example, early imple- Once the test effectiveness prob- sized and all the later activities are
test methodology based on solid sci- firm the hypothesis, and then uncovering new defects was low. mentation of HBT yielded significant lem was solved, the yield of defects in- about proving that the hypothesized
entific principles, It is like a Sherlock prescribes the treatment regimen,” As the product became huge and results for a global chip major. STAG creased, the focus shifted to defect(s) do or do not exist,” explains
Holmes-ian approach to testing - hy- quips Ashok. “It’s a personal test the company decided to re-architect it had to set up an effective validation streamlining the process by setting up Ashok.
pothesize potential defects and then methodology that fits any develop- in order to enable rapid feature addi- practice for their video decoder soft-
scientifically construct the strategy, ment methodology and weaves into tion with low risk. That is when the ware. The challenge that the company Optimization & reduction
test cases, measures, and tooling. HBT any organizational test process. The company decided to have a re-look at faced with their complex product that Test reengineering
Product Maintenance test
is a personal test methodology con- business promise of HBT is to slash its test assets and re-architect the same involved both hardware and software
sisting of six stages of cleanliness. It test and support costs and accelerate to increase the test coverage, improve and later system integration on multi- Quality enhancement
is powered by STEM (STAG Test En- LSPS solution
The tesing methodology is applied to
development.” defect finding ability, and ensure that ple realtime OS on various platforms Coverage enhancement
gineering Method), a scientific test en- was that of high defect escapes i.e. Productivity enhancement
validating products, requirements, and
Tool adoption
gineering method consisting of eight post-release field defects. Diagnostics & control
Evaluation suites
personal thinking disciplines aided by Organization E-Learning validation suite
architecture, quite different from the typical
Ashok and his team spent about a UT assessment
Mobile app validation suite
thirty two core concepts that aid sci- month understanding their domain and IT assessment
ERP validation suite
review or inspection process
entific enquiry. Evaluation of a soft- the associated technologies. Post this, Skill enhancement System enhancement mentt Bluetooth validation suite
COMPASSTM
ware or system in HBT consists of a detailed analysis yielded interesting DevQ system
People Skilling
Finishing school QA system
EBA Test services
setting up cleanliness criteria, then data – test cases were primarily con- Outsourced testing
CorpTraining Managed QA
identifying potential defect types, stag- formance oriented, coverage of test STEMTM Hypothesis BT Quality injection E&T JumpStart QA
ing them in an optimal order to create HBT has been applied by STAG in the test assets were future-proof. It had cases was suspect, escaped defects Retail Robust TD Req validation
ation
i Assessment services
Purposeful strategy
Successful automation Archi. validation
ti o nTA Custom tooling
a cost-effective staged evaluation various applications in areas like Mo- about 8,000 test cases then. seemed to propagate from early stages, Functional automation
model, formulating the various types bile, Healthcare, ERP, Media, eLearn- The company approached STAG and finally the process of validation LSPS validation
Boutique solutions from STAG Software
of test and corresponding techniques, a ing, and Supply Chain Management to analyze the existing test cases for was loose. Having understood the
formal design process to ensure that over the last nine years. This has re- completeness and modifiability and types of defects that were being found
the test cases are sufficient, and then sulted in lowered defects escapes (up re-architect the same after filling the and the post-release defects, they fig-
automating them as needed. to 10x lower), increased test coverage gaps and to ensure that the future test ured out the various types of probable proper gating in the test process and One of the USPs of HBT is its sim-
“The core theme of HBT is hy- (at least 3x), better RoI on automation, cases were easily pluggable. Applying defects and the various combinatorial creating a centralized Web based test plicity. Since it is a methodology gov-
pothesizing potential defects and then lower support costs (by 30 percent) STEM, STAG performed a thorough aspects that needed to be considered to repository, and finally setting up a erned by 32 core concepts and
scientifically constructing the strategy with no increase in effort, time or assessment of the existing test assets form a test case. The team then staged strong defect analysis system based on disciplined processes, it is easier to
and test cases, measures, and tooling. costs. HBT has been implemented in and discovered holes in the same. the validation as consisting of three Orthogonal Defect Classification learn and defies the common notion
It is a goal-centered methodology various process models including Using the STEM Test Case Architec- major levels, the first one at API level, (ODC) method. This enabled a strong that effectiveness in testing depends
wherein the goal of software cleanli- Agile. HBT has not only been applied ture (STEM-TCA), it re-engineered the next one at a system level, and the feedback system, resulting in shifting on the experience of the test staff. It is
ness is set up (i.e. a collection of clean- to validating products, but has been the test cases by first grouping them last one made up of a customer-centric the defect finding process to earlier a general belief that the more experi-
liness criteria), potential defect types applied to validating requirements and into features, then by levels of tests level that involved using reference ap- stages of SDLC and thereby lowering enced people in the QA team is the
that can impede the cleanliness crite- architecture, quite different from the and segregating into various types of plications. cycle time. Complementing this, better for the quality of work. At
ria identified, and then activities per- typical review or inspection process. tests, and then finally by separating Applying the HBT approach to test STAG focused on setting up a custom STAG, teams are not merely assem-
formed to ensure purposeful testing HBT has also been applied in assess- them into positive and negative test design, the test cases were developed, tooling framework for automating this bled based on number-of-years-of-
that is indeed effective and efficient,” ment of test assets to ascertain effec- cases. During this process of fitment yielding about 6,000 test cases at level non-UI based software, resulting in a experience, but on what key
explains T. Ashok, CEO, STAG Soft- tiveness and improve them. of existing test cases into the STEM- one and about 800 at the subsequent significant cycle time reduction – an disciplines are needed to perform the
ware. Based on sound engineering One such example is of a company TCA, STAG uncovered quite a few levels. Also, whereas the ratio of +ve entire cycle of tests on a platform took role and therefore the required compe-
principles geared to deliver the prom- that provides online banking solution. holes. These were filled by designing vs –ve test cases was earlier towards only less than 15 hours of time. tency. STAG evaluates the fitness of a
ise of guarantee of cleanliness, its core The company has three major products 5,000 test cases additionally. Not only the +ve side; after their re-design, the STEM is the basis for intelligent person to a team based on his or her
value proposition is about hypothesiz- catering to over 100 top financial in- did the STEM-TCA increase the test ratio shifted to 60:40 percent at the testing to deliver compelling business role and therefore the required compe-
ing potential defects that may be pres- stitutions (FI) including the top five coverage by uncovering the missing lower and about 85:15 percent at the results as it enables early detection of tencies to perform the role. Measuring
ent in the software and then allowing FIs in the world. It has a successful test cases, it also provided a sharper higher levels. Moreover, the number defects and enhances ability to find de- the increase in a QA’s competency
one to engineer a staged detection product line, growing rapidly with visibility of the quality as the test cases of test cases increased significantly by fects by at least three times, resulting while working with HBT, STAG de-
model to uncover the defects faster major releases almost every year, in- were well organized by specific defect a factor of 1,000 percent, allowing for in reduction of cost and time by over rived a competency model – CREAM.
and cheaper than the other typical test corporating new features to cater to the types. This improved the test coverage a larger and deeper net to catch many 30 percent. CREAM enables it to evaluate an
methodologies. various needs of the market place. As by about 250 percent and the technical more serious defects. Over the next 9 “We establish a clear goal and individual’s test-related competencies
“The concept of applying HBT on the code base evolved, the test assets management staff was confident about months, the rate and number of defects then perform activities that will indeed in terms of test lifecycle activities,
software is similar to visiting the doc- were also modified to reflect the the adequacy of test assets and was detected increased dramatically, re- get us to the goal. The goal translates technology aspects, and business do-
tor where he or she hypothesizes po- changed product. The challenge faced also convinced about its future up- sulting in post-release issues reducing to ‘uncovering potential defects’. main aspects. Based on this model,
The Smart Techie
|34| October 2010 The Smart Techie
|35|October 2010

4. SO WHAT DO CLIENTS HAVE TO SAY ABOUT STAG SOFTWARE?
competency gaps are analyzed, appro- he believes is quite different from a software program when it is written. Avionics, BFSI, Consumer Electronics,
priate training plan for the individual those of other vendors that were Traditionally, software testing, de- eLearning, ERP, Healthcare, Logistics,
is formulated, and the person is founded at the same time. pending on the testing method em- Mobile/Wireless and CDMA, Realty,
trained. 5 "& ,! , % &1 "*%+ ' -+ '& '+, ( * * +'-* -, "& %1 0( *" & ,!"+ "+ ,! Most testing services players com- ployed, can be implemented at any Semiconductor, Shipping, and popular
Engaging the Customers /*'& % ,*" ! * $ # 1 ,' + $ ,"& & ' +!'* . $'(% &, ( *,& * "+ (*' pete on the ‘volume game’. Every time in the development process. Internet search engines – STAG truly
As a method oriented organization, the - ,".",1 ( * '$$ * '* ('-& . *+-+ '+, * +'-* ' ,!"+ 0, &, / ! . '-& project is won on the basis of the size However, most of the test effort occurs proves that its methodologies are do-
art of customer engagement is as ,' (*'." 0 (,"'& $ . $- 6 of the team or number of service loca- after the requirements have been de- main independent. The ultimate recog-
unique as the company itself. Since the *,". "* $ ++ tions they can offer to their customers. fined and the coding process has been nition for the company is STEM 2.0
industry is yet to fully understand the “This is the fundamental difference completed. As such, the methodology being featured in the Unisys Technol-
company’s approach, STAG first edu- between STAG and other independent of the test is governed by the software ogy Review, a well known journal in
5 % . *1 "%(* ++ /",! 4+ ((*' ! ,' , +,"& ! "* )- $",1 ' , +, +" &
cates its customers about the better testing vendors,” Ashok points out. development methodology adopted. the Japanese software industry and ac-
&'%($ , & ++ ' , +,"& + * +-$, ' ,! "* % ,!' '$' 1 "+ ." &, "& . *1 *
quality and RoI by deploying STEM STAG essentially is a ‘methods’ “As software services companies ademia, and part of the Japanese Na-
$ + ! '%%",% &, & $ 0" "$",1 +!'/& 1 ,! , % ,' '%%' , * +
and HBT. At present, STEM and HBT company and not a product or services graduate from plain coding work to tional Archive. “Japan is a research and
are primarily deployed by the com- +". * $ + , + "+ (* "+ /'*,!1 6 one. The core foundation of the com- full development (not necessarily de- technology oriented industry and it is a
pany’s customers only in certain parts &# 2 * '% pany lies in the fact that it develops veloping shrink wrap products) testing huge honor to be recognized by them,”
of a project. Ashok has also started to methodologies or concepts that can be becomes crucial. But unfortunately, Ashok smiles.
license the methodology to companies 5 ,*- ( *,& * ,' ! + /'*# /",! $%'+, $$ ,! *'-(+ & ! + $+' used in testing irrespective of the do- many programmers are forced to do With its path breaking research ori-
who wish to deploy it internally and (*'." '-,")- +'$-,"'&+ $"# +, +" & + *." + & & ,' & -,'% ,"'& ' main or technology. minimal testing to beat the last minute ented approach to software testing,
customize it to their requirement. ,! * "& *" -+"& & *$ 6 This ideology of working on test- project deadline,” he says. STAG has managed to disrupt the way
Start with establishing a clear goal and then
These include software development *' * % & * ing methods was influenced by IBM
perform activities that will indeed get you to
companies and other independent Rational that had worked long years to
service providers amongst which is 5 ! + $/ 1+ * +('& +/" ,$1 ,' '-* * (" $1 *'/"& * +'-* & + '* , +, set up methods for software develop-
translating the goal into ‘uncovering
one of the top five Japanese system in- "& ! "* $ 0" "$",1 ,' &+-* -+"& ++ '&,"&-",1 "+ !" !$1 ((* " , ! '%%", ment and over the years came up with
potential defects’
tegrators that today deploys STEM as % &, & ,! $ 0" "$",1 +!'/& 1 ,! "* , % ,' % , !" !$1 * ++". * $ + a standard called UML (Unified Mod-
a part of its services. ,"% $"& + "+ '%% & $ ! . $/ 1+ '-&, '& & /"$$ '&,"&- ,' eling Language). Today, UML is the
“We are very serious about the ' +' 6 basic framework around which soft-
economics of testing and its contribu- * & * % * # *
ware is developed despite what the
tion to the bottom-line of our cus- technology may be. It was the lax approach towards testing has been practiced over the
tomers. At every instance, we question A Masters in computer science testing and the challenges faced by decades. “I believe this is quite an
5 ' -+ '& + " &," " / 1 ' '"& , +,"& /",! ,! "* % ,!' '$' 1 / +*
ourselves the value being delivered from the Illinois Institute of Technol- quality analysts that sparked Ashok’s achievement but there is still some time
$ , "& ,! ,". & ++ ' , +, *," ,+ * , 1 ,! "* , % ! '%%",% &, '
and how it has impacted our cus- ogy, Ashok had always had a knack for thought process: could there be a before the industry as a whole adapts
tomer’s business,” says Ashok. When + &"'* % & *+ "& * ." /"& & % &, ! $,! ( *"' " $$1 & , #"& ((*'(*" , research oriented approach to prob- methodology for testing that could it,” says Ashok. In order to evangelize
engaging with a customer, STAG pays +, (+ , ! +, * $$1 ! $(+ "& ,,"& +, '* ,! "&. +,% &, / % # "& lems. It was while leading the world- work as a scanner wherein if one ran a the adoption, STAG itself has taken the
a lot of attention to understanding. +'$-,"'&+ 6 3 wide software test analysis group piece of code, it could immediately initial steps by offering a full fledged
HBT takes a scientific approach to the ! "*% & & '-& * -* / . + (WW:STAG) at Verifone India, for- highlight the bugs. Thus, he came up education division to train people in
act of understanding the intentions or merly a division of Hewlett Packard, with the concept of hypothesis based STEM and HBT and make them indus-
expectations by identifying key ele- ment, people competency assessment ing industry, it is true that most entre- that he started dabbling with the idea testing. try ready. The programs offered from
ments in a requirement or specification (in testing), test-case re-engineering, preneurs have shied away from getting of finding more effective methods for Today, a decade since its beginning, its Bangalore and Chennai facilities
and setting up a rapid personal process custom tooling for test bench, require- into the business testing. Industry testing. STAG has successfully worked with have already proved quite effective as
powered by scientific concepts to ment and architecture validations, in watchers attribute two probable rea- “WW:STAG was a large inde- over 100 customers and conducted a majority of the trainees are being ab-
quickly understand the intentions and addition to standard offerings like out- sons for this - one, the general mis- pendent test organization in Veri- around 350 cycles of various types of sorbed by the company’s clients them-
identify the missing information. This sourced QA Lab, managed validation, conception that testing, unlike fone/HP responsible for certifying test viz., functional testing, load/perfor- selves. Also, he plans to publish all the
enables the engineers to come up with functional test automation, and per- software services or products business, client server and electronic commerce mance/stress/volume test, reliabil- research on STEM and HBT and bring
intelligent questions leading to a formance assessment. is not a lucrative proposition and the software. This group built ground up a ity/scalability test, test automation, Beta out a book soon that could perhaps one
deeper understanding. This results in other is the feeling that testing would test group well recognized within and testing, Installation testing, configura- day be an academic reference in soft-
rapid ramp-up, higher business value Developing a Methods Company figure at the low end of the market. outside the company. It was the ideal tion/compatibility testing, L10N test- ware testing.
being delivered, and leveraging this Software testing, though critical in Neither of these fears is based on platform for me to experiment on the ing, document testing, API testing, and Ashok often quips that software
understanding to build a rich asset software development, for years has facts, says Ashok whose passion for scientific approach to testing,” says more. testing is like the ‘ugly duckling’ that
base. been considered the most jaded and testing resulted in STAG Software. Ashok. Having worked on 170+ projects in not many prefer to step into, but it is
STAG offers unique solutions and mundane task in software develop- Today, STAG has firmly positioned it- As the software becomes complex the areas of enterprise applications, em- just a matter of time that it will trans-
services like test assessments, valida- ment life cycle (SDLC). Though India self as a boutique company offering day by day, there is also an increase in bedded software, and test automation form into a beautiful swan and hope-
tion suites, release worthiness assess- has carved a niche for itself in the test- testing solutions and services, which bugs or software defects that slip into with customers spanning industries - fully STAG will be the catalyst.
The Smart Techie
|36| October 2010 The Smart Techie
|37| October 2010