Testing
Upcoming SlideShare
Loading in...5
×
 

Testing

on

  • 361 views

 

Statistics

Views

Total Views
361
Views on SlideShare
361
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Testing Testing Document Transcript

  • Testing Times February 2003 Testing Times No 18 February 2003 The IPL Software Products Newsletter No 18 IPL Launches Software Testing Service IPL is pleased to announce the launch project definition, requirements platforms, the HTML Results Viewer of its Software Testing Service (STS). analysis, design, implementation and adds a new colourful dimension to We can now offer customers a (of course) testing. IPL is an approved reviewing test results and coverage tailored software-testing package. defence contractor and is therefore able reports. The service will be of particular to undertake classified government AdaTEST 95 interest to developers of high- work under the STS. IPL's QMS is A multi-platform user interface with integrity systems who are required to certificated to ISO 9001:2000 and to wizard driven scripting and HTML meet the demands of various TickIT, assuring our customers of an Results Viewer are now delivered as software development standards. auditable, high-quality testing process. standard with AdaTEST 95. IPL will STS will allow developers to For further information, please contact soon be releasing a new version of concentrate on the more productive sts@iplbath.com. AdaTEST 95 with an Ada 95 Test processes in their software lifecycle Support Package (TSP) generator for and subcontract the labour intensive parts of the testing process to IPL. Product News the tool. TSPs are package-specific testing declarations and procedures, The STS is available from February Cantata++ which greatly enhance the productivity 2003. Developers will initially be Cantata++ V 3.0 has been released with and quality of user-generated test requested to submit their testing the following new features: scripts. Ada 83 customers migrating to requirements, and a sample of source ! Multi-platform user interface with Ada 95 can now also benefit from a code and design specifications to IPL wizard driven scripting ; migration tool to convert their for analysis. This will allow us to scope ! Full compatibility with legacy Cantata AdaTEST .ts scripts to a human- the problem and provide a fixed-price scripts; readable format, compatible with for the work. The STS is available for ! HTML Results Viewer; Ada95 Test Support Packages (TSPs). any level of software testing and for ! Modularised Test Harness for scaleable software written in any language or platform - not just the languages and embedded target support. Bugz Bunny This makes it, we believe, “probably the platforms supported by IPL’s tools. Continuing the ongoing catalogue of best C/++ testing tool in the world”, software glitches that should have been S h a u n D a v e y, I P L’s S o f t w a r e offering support for ANSI C, C99, ISO tested before their presence became Technology Manager said “We are very C++, and EC++, in one package on a public knowledge! excited that we can now offer this new variety of host and target platforms. service to our clients, many of whom June 2002: Renault cars (Laguna 1.6) at are overwhelmed by the testing effort risk due to ‘software faults’ that could required by the high-integrity software lead to engine cut-out at low revs. development standards. Clearly, the August 2002: Software failure distraction of ‘jumping through the ‘possibly’ at root of failure of $8.7m hoops’ of the standards can actually Japanese SST rocket prototype, by detract from the quality of their causing premature separation of upper software. We aim to allow our clients to stage from lower. subcontract a substantial part of the September 2002: First launch of the testing effort to us, allowing them to Delta EELV rocket delayed by a month concentrate more on the functionality due to a software fault in the last minute and quality of their final system. of the countdown sequence. Another key advantage of STS is IPL's independence: many of the standards November 2002: A ‘software glitch’ in used by our clients call for Germany led to the bills for phone taps independence between the being sent to the alleged criminals development and testing teams.” instead of being charged to the security services which had ordered them. The In addition to providing the STS, IPL is mistake was only discovered when the HTML Results Viewer (HRV) able to assist with all aspects of the targets complained about the mystery Available free with all maintained software lifecycle: initial conception, items on their telephone accounts. product licences on all standard Inside: AdaTEST 95 in Space Systems Article on Embedded C++ All rights reserved Beagle 2, www.beagle2.com
  • February 2003 Testing Times No 18 AdaTEST 95 at Astrium “There’s No Room For Failure” Astrium is Europe’s major satellite and space vehicles developer. It has operating companies located in France, Germany, UK and Spain, and is a successful participant in space work all over the world. Two current projects in the UK serve to illustrate both the breadth of Astrium’s work and its commitment to producing on-board software to the demanding standards required for deployment in space. Funded by the Canadian Space Agency, RADARSAT-2 is planned to launch in 2004; it will carry advanced radar All rights reserved Beagle 2, www.beagle2.com equipment enabling it to perform detailed topographic mapping of the Earth's surface. The Synthetic Aperture Radar Sensor Electronics it will use is being supplied by Astrium Ltd, based on equipment and systems originally developed for Europe’s Envisat. A different kind of spacecraft altogether, Beagle 2, is a UK-led Mars Beagle 2 on the Surface of Mars lander, intended to carry out a range of experiments to investigate whether life particular the high-reliability software development work was Dan has ever (or still does) exist on Mars. It ‘Ravenscar profile’ subset. Another Marshman. He says that the ease with will be launched in June 2003, as part of decision was to conduct a detailed which his engineers learnt to use the larger ESA Mars Express mission. program of software unit testing. Dave AdaTEST 95 was a crucial factor in Beagle 2 is being developed by Astrium Yetton, Software Project Manager for making the team productive quickly. Ltd. on behalf of the Open University RADARSAT-2 says, “As a company we “In a matter of days, a programmer who are leading the academic are 100% committed to the idea of unit could be testing code with AdaTEST 95 investigations of the quest. testing. We have done it before and from a standing start. The product needs believe it is one of the best ways to gain little or no training and is quite intuitive Maximising Reliability confidence in our code.” to learn, even for beginners.” The total Both projects require the production of volume of RADARSAT-2 code that large amounts of on-board software. The choice of software testing tools was fairly quickly resolved because Astrium needed testing was around 20 KLoC The need for the highest level of with about 225 separate unit tests to be software reliability is paramount in both had used AdaTEST previously on the Envisat programme and other projects. created and run. Given that the cases, though for different reasons. In Ravenscar target environment was a the case of the RADARSAT-2 project, AdaTEST had a good track-record in the company and was liked and new quantity for Astrium, it was the need for reliability is driven by important to ensure that tests would also customer requirements for high- respected by Astrium’s software engineers. In the case of Beagle 2, run on the target. This was facilitated by availability of the sensing instruments. AdaTEST 95's target features - scripts A significant proportion of the software Astrium were not directly involved in writing the software, but were using two can be run unchanged in the target cannot be changed once the craft is environment. It became the project launched, so the very highest standards software subcontractors (Logica and Coda SciSys). Bill Edwards is standard that all ‘critical’code would be have to be applied. In the case of Beagle tested in this way, while less critical 2, the weight constraints on the craft are Astrium’s Software Manager in charge of the Beagle 2 programme, and says, code would be unit tested only on the such that very few of the usual host. redundant systems can be incorporated “Both of these companies had previous in the design. Hence the systems, positive experience of AdaTEST, so the The total size of Beagle 2 code was including the software, just have to choice of AdaTEST 95 was easily about 950 Kbytes with about 80 work - there is no fallback. agreed.” Delivery of the tools took Objects. The original intention had been place during early 2000, and work to test only on the host but it very Quite early on in the software projects’ began. quickly became apparent that a lot of lifecycle some key decisions were taken the code could only be run on the target. to help maximise software reliability. Testing Productivity The Astrium engineer most closely Since the initial AdaTEST 95 delivery One of these was to use Ada, and in did not permit coverage on the target involved with the RADARSAT-2 2
  • February 2003 Testing Times No 18 (due to the Ravenscar restrictions of the Conclusions applications. The resulting EC++ compiler) a minor crisis threatened to Both the RADARSAT-2 and Beagle 2 standard was designed to take develop. However, says Bill Edwards, projects are nearing the end of software advantage of C++’s most useful “IPL rose to the occasion and offered to production. Bill Edwards has only features, but at the same time discarding look into the problem based upon an praise for the contribution of AdaTEST those features responsible for code idea provided by the compiler supplier. 95. “It gave no problems in use for the bloat, such as multiple inheritance, They quickly implemented a very entire duration of the project and we exceptions, runtime type information capable working solution, which most could carry out coverage testing on the and virtual method tables. importantly for us did not eat into our target with the updated version, even Testing EC++ limited budget and has proved as though the compiler supported the Traditionally, testing solutions are reliable as the rest of the AdaTEST 95 Ravenscar Profile.” Dan Marshman implemented in the form of reusable product.” concurs, “It was productive from day 1. test ‘libraries’ of C++ routines. These AdaTEST 95 has been fantastic for us. help a tester by providing a structured It is easy to use, reliable, and gives us framework in which to carry out testing exactly what we want, both from our in a controlled, repeatable, auditable © Canadian Space Agency, www.space.gc.ca own point of view as developers, and manner. For testing to be performed on also looking at the need to give our the target, these libraries must have customers results data which they can been compiled with the target compiler. understand and depend on.” The usual method of dynamic testing is Dave Yetton has the final word. “My to write a test script (in C++), which projects have used the IPL testing tools uses the test library routines, and calls for quite a number of years. Over that the software under test. This script is RADARSAT-2 period we've come to respect the then compiled with the source under New Ways of Testing products, and also IPL’s willingness to test and linked with the test libraries, The RADARSAT-2 project was keen to support them and provide us, the and any other libraries required by the explore new ways of testing which customer, with an exceptional level of software under test. would build on the previous assistance. I fully expect that we will be Memory and performance issues apply experiences of AdaTEST. Dan using more of IPL’s tools in future as much to dynamic test code as they do Marshman describes some of the work.” to released code. Any dynamic test innovations they introduced. “The first IPL would like to thank Astrium for script must itself conform to the improvement we made was to use permission to report on their embedded subset used for shared stubs. In this way we avoided experiences with AdaTEST 95, and take development, because it is to be many of the recompilation issues this opportunity to wish them well with compiled using the embedded compiler. introduced previously as code changed. ongoing and future programs. C++ features not supported by EC++ The vital enabling technology was our must also be disabled in test libraries. configuration management system which allowed AdaTEST 95 scripts and C++ for embedded Much of the code bloat in C++ does not come from using features such as stubs to be kept in tight step with each applications templates, but from referencing other.” In this short article, Neil Langmead templates found in large C++ libraries. A second major issue was that they were reviews the use of embedded C++ Including such references often means making extensive use of OO design and (EC++) and considers the testability of compiling and linking-in many parts of programming techniques. The this language. the library that simply aren't needed. approach to testing that the team Has embedded C++ proved to be the Another testing problem to consider is adopted was based closely on the panacea for C++’s problems in the scalability of EC++ hierarchical integration testing methods embedded environments? C++ is one of implementations. A further ‘standard’, recommended by OO specialists. This the most elegant and sophisticated ETC++, has been developed. This involves fully testing code for base programming languages available to takes the baseline EC++ definition, and class implementations and then testing the developer. The downside to C++’s scales it, adding support for more of the only the added or changed functionality rich feature set is that compiled object original ISO C++ features. Many for derived classes. “This worked well code can be several times larger than the compiler vendors offer ETC++ support, for us,” says Dan Marshman, “even equivalent code compiled in C, leading with the ability to switch on or off though we had up to five levels of to increased memory requirements and individual language features at will. derived classes.” As work progressed a reduction in execution speed. On host (E.g. templates and exceptions). As PJ they also learnt another useful OO systems, these are perhaps of Plauger, the man who introduced EC++ testing trick, namely that of coding tests superficial concern - the benefits of to the world, puts it: “Nobody wants a as child units of the units being tested. C++ (component reuse and faster proliferation of dialects. The whole idea This conveniently allowed white-box development) usually outweigh behind the formation of the consortium access to class private components, and performance issues. For embedded was to avoid such mayhem in the is a useful addition to the more well- applications though, memory and Japanese embedded systems known test points technique for execution speed are critical. com munity. N evertheless, one accessing package body data. They did significant new dialect [ETC++] has learn however that Ada protected EC++ as an industry ‘standard’ evolved out of discussions among Japanese already appeared. And it could become objects do not fit comfortably into a at least as popular as EC++.” deep class hierarchy, so some new automotive manufacturers, who recognised the need for a subset of C++ This pr olif er ation of subset s, techniques had to be evolved to deal that performed well for embedded particularly when compiler vendors with these situations. 3
  • February 2003 Testing Times No 18 were struggling to achieve successful ISO C++ implementation, has not aided New Customers Meet Us testing. The ability to scale C++ code at In the last few months we have been Here are some of the forthcoming will requires similarly scaleable grateful to receive the custom of the events where you can meet us: dynamic test harness solutions. following new companies (in Safety Systems Symposium (SSS'03), Solutions alphabetical order): African Defence February 5-6, Bristol Cantata++ solves the above problems Systems, C&C Electronics, Ceprei ICS Test,April 2-4, Cologne, Germany of code bloat and scalability, through its Software Test Centre, Cognizant Te c h n o l o g y, D e n e l Av i a t i o n , Embedded Systems Club, April 9-10, powerful and highly configurable Swindon E[T]C++ compatible test harness. The EastCOM, Hamilton Medical, Medical libraries are implemented very Intelligence, Nanjing Institute of AdaEurope, June 17-19, Toulouse, efficiently, minimising the code bloat Electronics, Samsung, and Shanghai France that blighted many embedded C++ Institute of Control Engineering. Joint seminars with GreenHills and applications in the past. With the i-Logix, provisionally April 29 and Cantata++ libraries, individual features Customer Quotes May 1. such as templates, exceptions, RTTI, “There's a lot of complicated dynamic Wchar_t, and timing can all be switched data structure / link list stuff in my IndianAgent on or off, according to which C++ current work and Cantata has made features are being used. Additionally, IPL is pleased to announce that AK testing this a breeze” TTP Comms Cantata++ uses the industry leading C Aerotek will act as IPL's local agents in “Glad to get such a prompt reply from India. For contact details of all IPL and C++ parser from Edison Design you. It is seldom that we have such Group (EDG). PJ Plauger comments: distributors and sales agents see our prompt support.” Infosys website at www.iplbath.com/tools/ “EDG supplies the C++ front end for many embedded C++ compilers, and it “I believe I'm the one in debt here. partners. is at the leading edge. It even has dialect Without AdaTEST I would have had a switches already installed for tailoring miserable time ... Having worked with Training the recognized language to be EC++ or a few other test tools since ending my We have introduced some new training many other interesting variants.” contract AdaTEST is still my courses to accommodate the newly favourite.” Aconsultant, Norway revised products. The introductory ETC++ seems a happy compromise between ISO C++ and EC++, giving the “I am pleased to recommend IPL as an courses are now: developer access to the popular STL approved supplier of Cantata and the ! Cantata++ for C++ template library. With significant high integrity package to … The ! Cantata++ for C memory savings of up to 50% on the full development and subsequent support of ! AdaTEST 95 ISO C++ implementation, widespread the tool has been to a high standard in accordance with a mature Quality All courses are of one-day duration. We acceptance among leading compiler expect to be introducing several half- vendors such as Greenhills, plus an System which more than satisfies the requirements for tool validation as day specialist training modules over the excellent testing solution to support it, next few months. For further details see: ETC++ looks to be the perfect choice defined by RTCA DO 178B.” Recent Cantata audit report. www.iplbath.com/tools/training. for embedded developers. Cantata++ fully supports EC++ and ETC++. Users considering testing these “I used AdaTEST (with MUCH success) in the F22 project… I am now Contacts working on Ada code to support the IPLSoftware Products Group C++ variants should contact Neil Eveleigh House, Grove Street, Bath Langmead (neil.langmead JSF/F35 program. I IMMEDIATELY BA1 5LR, UK @iplbath.com) for more information. suggested AdaTEST.” A Lockheed Tel: +44 (0) 1225 475000 Martin engineer. Fax: +44 (0) 1225 444400 The full transcript of this paper can be Email: tools@iplbath.com found at http://www.iplbath.com/pdf/ p0835.pdf. New Article www.iplbath.com/tools For an excellent description of the use Monthly News of AdaTEST 95 in a Linux/GNAT environment please see Distributors Don't forget that a monthly news update NorthAmerica: Quality Checked Software www.linuxjournal.com/ Contact: Scott Thomas, cst@qcsltd.com appears on our website, at article.php?sid=5965. Our thanks to the Tel: +1 503 646 9991, Fax: +1 503 646 8783 www.iplbath.com/tools/news. author, Thomas Osterlie of ConsultIT. China: Watertek Contact: Cai Houfu, caihoufu@watertek.com Testing Ted Gilchrist & Downing Tel: +86 10 8288 3933, Fax: +86 10 8288 3858 4 All trade marks acknowledged