Testing parallel programs


Published on

Testing parallel software is a more complicated task in comparison to testing a standard program. The programmer should be aware both of the traps he can face while testing parallel code and existing methodologies and toolkit.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing parallel programs

  1. 1. Testing parallel programsAuthor: Andrey KarpovDate: 02.11.2008AbstractTesting parallel software is a more complicated task in comparison to testing a standard program. Theprogrammer should be aware both of the traps he can face while testing parallel code and existingmethodologies and toolkit.IntroductionSpeaking about software development we single out the step of testing and debugging an application.The notions of testing and debugging are understood as nearly the same. It is due to that after an erroris found during the process of testing its correction often relates to the process of debugging of aprogram. And debugging of an application can be in its turn called "white-box" method testing.Sometimes the step of debugging is understood as simultaneous search and correction of errors.Still, well distinguish between these two notions and concentrate on testing parallel software in thisarticle.Testing software is a process of detecting errors in software by using different tools and analysis ofprogram workability by end users [1].Debugging is a process of finding errors by the programmer in the code which were detected whiletesting the software to correct them. Debugging usually implies using specialized tools for tracking theprograms state while executing it.Debugging parallel programs is a thankless task demanding accuracy and use of specialized tools. A lotof articles are devoted to debugging parallel programs and even more should be devoted as this topic isvery urgent because of rapid development of multicore systems and new technologies of creatingparallel programs. In the sphere of toolkits there are also some problems. But before you could performdebugging you should find the error. Meanwhile some debugging methods not only find errors but alsodetect their locations. Thats why lets consider testing.1. Parallel program testing difficultiesTesting applications with parallel processing is not a simple task. Paralleling errors are difficult to detectbecause of parallel applications non-determined behavior. Even if an error is detected it is often difficultto reproduce it. Besides, when the code is modified it is not so easy to make sure that the error is reallycorrected and not concealed. We can give it another name, that is the errors in a parallel program aretypical "heisenbugs".Heisenbug is a term used in programming to define a program error which disappears or changes itsproperties when trying to detect it [2]. This name is a pun and comes from a physics term "Heisenberguncertainty principle" which is understood in everyday life as a change of an object being observed
  2. 2. occurring in quantum mechanics. An example of such errors is errors which show up in the final programversion ("release") but cannot be seen in debugging mode; synchronization errors in a multithreadapplication are another example.Thus, the task of parallel testing on a whole comes to the problem of creating diagnosing tools whichminimally influence a programs behavior or creating all the necessary conditions for it to occur. Thatswhy lets look at the classic methodologies from a new viewpoint.2. Methods of searching errors in parallel programsMethods of searching errors in parallel applications as well as in sequential ones can be divided intodynamic analysis, static analysis, model-based test and programs correctness proof [3].Formal proof of a programs correctness is a very complicated procedure especially in the sphere ofparallel programming and it is usually not used in industrial software development.Dynamic analysis implies the necessity of launching an application and executing different sequences ofoperations which aim at detecting incorrect program behavior. A sequence of operations can be definedboth by man during manual testing and when using different tools implementing load testing or, forexample, check of data integrity. In case of parallel programs of the most interest are tools like IntelThread Checker which provide an applications source code with monitoring and logging means whichallow you to detect deadlocks (both explicit and potential), hangs, races etc [4].Static analysis handles only an applications program code without launching it. As an advantage of thismethod we can single out detailed and full coverage of the analyzed code. We should mention that inthe sphere of parallel programming static analysis method is used rather rarely and is represented byfew tools which are rather research tools than commercial products.Model-based test is automatic generation of tests according to the defined rules. Model-based testallows you to formally ground absence of defects in the code part being tested on the basis of the rulesof data conversion defined by the developer. Such an example is KISS tool developed in MicrosoftResearch for C parallel programs.All the mentioned methods have disadvantages and this doesnt allow you to rely only on one of themwhen developing parallel programs.Dynamic analysis demands launching programs, it is sensitive to the execution environment and itsignificantly slows down the speed of application execution. It is rather difficult to cover the wholeparallel code with tests. It is often possible to detect race conditions only when it occurred in the givenprogram session. That is, if a dynamic analysis means informs you that there are no errors yet youcannot be sure about it.In case of parallel applications static analysis is very complicated and it is often impossible to predict theprograms behavior as the acceptable set of input values for different functions and the way of callingthem are unknown. These values can be predicted on the basis of the rest code but within certain limitsas a huge space of possible states appears and the size of the tested information (variants) becomesunacceptably large. Besides, static analysis means often display a lot of false messages about potentialerrors and demand great effort to minimize their number.
  3. 3. In practice model-based test works only for small base blocks of an application. In most cases it is verydifficult to automatically build a model on the basis of the code while manual creation of models is avery laborious and resource-intensive operation. In fact, it is necessary to write the same dataconversion algorithms but in a different presentation. As in case of static analysis a problem of quickextension of state space appears. State space extension can be partially controlled by using methods ofreduction with complex heuristic logic but it leads to that some defects will be missed. An example of atool for testing parallel application projects on the basis of models is Zing.As we see, using only one of the approaches to testing parallel applications is inefficient and it would bea good solution to use several methods for testing one and the same program product.3. New technologies - new toolsAs the choice was made for benefit of multicore processors when solving the task of performanceincrease this resulted in a new round of development toolkit advance. For multicore systems withcommon memory it is more convenient to use such programming technologies as OpenMP instead ofmore habitual MPI or standard paralleling means provided by operational systems (fork, beginthread).New technologies need new tools for supporting them. Now we should attentively follow the newsystems of parallel programming support appearing on software market. They can significantly simplifyyour work and help you to quicker adapt your applications for efficient use of the parallel environment.One of such tools is the static code analyzer VivaMP developed by OOO "Program Verification Systems"company.As was said before static analysis of parallel programs is complicated and ineffective as it is necessary tostore too much information about the possible program states. This is absolutely true in case of usingsuch parallel programming technologies as MPI or paralleling by operational systems means.It is better in case of OpenMP technology and you can often implement efficient static analysispossessing good characteristics. This is due to that OpenMP technology is meant for paralleling isolatedcode sections. OpenMP, so to say, allows you to make a program parallel "in small parts" by puttingspecial directions in the most performance-critical code sections. As the result the parallel code appearsto be grouped and it doesnt depend on other applications parts what allows you to perform a high-quality analysis.Until recently the area of static analysis of OpenMP programs has not practically been developed. Wecan give only one example, that is Sun Studio compiler performing rather good diagnosis. VivaMP staticanalyzer has occupied this sphere. This is a specialized tool for searching errors in parallel programsdeveloped with the use of OpenMP technology in C and C++ languages [5].This analyzer both detects explicit errors and informs about potentially unsafe code. To show theprocess of error diagnosis lets consider an example of detecting use of one variable for saving fromparallel streams without necessary synchronization::int sum1 = 0;int sum2 = 0;#pragma omp parallel for for (size_t i = 0; i != n; ++i)
  4. 4. { sum1 += array[i]; // V1205 #pragma atomic sum2 += array[i]; //Fine }And to show the process of potentially unsafe code diagnosis lets consider an example of using flush fora pointer. Its easy to make a mistake and forget that it is the pointer to which flush operation will beapplied and not the data it refers to. As the result the given code can be both correct and incorrect:int *ptr;...#pragma omp flush(ptr) // V1202int value = *ptr;In the next VivaMP analyzers version some tests will be implemented which relate to detectingineffective parallel code. For example, there will appear critical sections where it would be only enoughto use the quicker directive atomic:#pragma omp critical{ a++; //Slow}#pragma omp atomica++; //GoodConclusionDifferent forms of paralleling have existed in the software world for a long time. But to create masscommercial applications which would use the possibilities of multicore processors to full extend weneed other development means different from those used when creating sequential applications. Wehope that this article will throw light on all those difficulties relating to development of parallelapplications and the programmer will be most serious when choosing appropriate means of developingand testing such programs.References 1. Wikipedia. Testing software. http://www.viva64.com/go.php?url=152 2. Wikipedia. Heisenbug. http://www.viva64.com/go.php?url=153 3. Rahul V. Patil and Boby George. Tools And Techniques to Identify Concurrency Issues. http://www.viva64.com/go.php?url=154
  5. 5. 4. Evgeniy Romanovski. Debugging of multi-thread OpenMP-programs. http://www.viva64.com/go.php?url=1555. Andrey Karpov, Evgeniy Ryzhkov. Adaptation of the technology of the static code analyzer for developing parallel programs.. http://www.viva64.com/art-3-2-486346462.html