Static Analysis
An Empirical Case Study
Aaron Kim
neohckim@gmail.com
2009.10.08

목표
소프트웨어 품질 향상을 위한
정적 분석 도구 활용 가이드

우리의 현실
• 정적 분석(Static Analysis) 도구
– 정적 분석?
– 장점과 단점은 무엇이고 비용문제는 어떠한가?
– 정말 필요한가?
– 무료 또는 오픈 소스는 없는가?
• 제시 방향
– 올바른 경험과 정보의 공유
– 소프트웨어 품질/개발 ROI 향상 대안

어떠한 솔루션이 있는가
그 외? 오픈 소스?…

상용 정적 분석도구
• 어떤 잣대로 정적 분석 도구를 평가 해야 하는가?
• 참고할만한 평가 결과는 공개되어 있는가?
• 각 벤더의 홍보 자료는 믿을 만 한가?

상용 정적 분석도구 평가
Static Analysis Tool Exposition

상용 정적 분석도구 평가
• NIST (National
Institute of Standards
and Technology)
• SAMATE (Software
Assurance Metrics
And Tool Evaluation)

소프트웨어 결함
• 소프트웨어 결함(Defect) 종류?

소프트웨어 결함을 체계적으로 구분?
List of Defects In Detail
Value(s) assigned incorrectly or not assigned at all; but note that a fix involving
Assignment/Initialization
multiple assignment corrections may be of type Algorithm.
Errors caused by missing or incorrect validation of parameters or data in conditional
statements. It might be expected that a consequence of checking for a value would
Checking
require additional code such as a do while loop or branch. If the missing or incorrect
check is the critical error, checking would still be the type chosen
Efficiency or correctness problems that affect the task and can be fixed by
Algorithm/Method (re)implementing an algorithm or local data structure without the need for requesting a
design change. Problem in the procedure, template, or overloaded function that
describes a service offered by an object
The error should require a formal design change, as it affects significant capability,
Function/Class/Object end-user interfaces, product interfaces, interface with hardware architecture, or global
data structure(s); The error occurred when implementing the state and capabilities of a
real or an abstract entity.
Timing/Serialization Necessary serialization of shared resource was missing, the wrong resource was
serialized, or the wrong serialization technique was employed.
Communication problems between:
Interface/O-O Messages 1) modules 2) components 3) device drivers 4) objects 5) functions
via
1)Macros 2)call statements 3)control blocks 4)parameter lists
Problems related to associations among procedures, data structures and objects.
Relationship
Such associations may be conditional
http://www.research.ibm.com/softeng/ODC/DETODC.HTM#overview

소프트웨어 결함을 체계적으로 구분?
“We conclude that the distribution of software faults
depends heavily on project-specific factors, such as the
maturity of the software, operating environment [LI95] or
programming language. Unfortunately, researchers have
not established the set of such factors yet, and it is not
obvious how one would systematically discover them.
The already long history of software fault categorization
efforts suggests that this task is not likely to be
completed in the near future.”
Jan Ploski, Research Issues in Software Fault Categorization,
ACM SIGSOFT, November 2007

소프트웨어 결함을 체계적으로 구분?
“The range of efforts to create defect classification
schemes described earlier in this paper, and the long
history, in which there has been no single, widely used
scheme, suggests that defect classification is
hard, and repeatable orthogonal classification is in itself
difficult.”
“It is hard work to develop a classification scheme that is
both complete and repeatable in the ideal sense.”
Diane Kelly and Terry Shepard, A Case Study in the Use of Defect
Classification in Inspections, Royal Military College Of Canada

분석 기술 In Depth
• 분석 기술
–어떠한 방법이 있는지?
• 주요 관련 conferences
– SAS(Static Analysis Symposium),
CAV(Computer Aided Verification), PCODA,
WDOA, ISSTA, SSV, VMCAI, SCAM, ESEC,
ASE, ICSE
–현업에서 적용 가능한 실용적이고
효율적인 방법인가? Well, Let me see…

분석 기술 Intro

주요 Notation
Nevin Heintze, Set Based Program Analysis , Degree of Doctor of
Philosophy, p.358

Static Analysis 기술 도표 일부
Available Expressions Analysis
Reaching Definition Analysis
(Optimistic)
Intraprocedural Analysis Very Busy Expressions Analysis
Live Variables Analysis
Data Flow Analysis
(Pessimistic)
Constant Propagation
Interprocedural Analysis
Shape Analysis
Constraint Based Analysis DEPTH DIRECTION
Abstract Interpretation
Type and Effect System
Principles of Program Analysis, Flemming Neilson

상용 정적 분석도구 결과
• 발표된 자료(논문)는 있는지? Yes
• Lesson Learned?
– 실력 있는 팀을 구성하여 개발팀과 Cohesive하게,
– 오픈 소스의 활용을 1차적으로 고려
– 상용 툴 도입 시 False Positive, False Negative
정책을 유연하게 가지고 가기
– 안정화 후, 소프트웨어 프로세스 化

운용 결과 참고 I
• The cost of ASA per detected fault is of the same order of magnitude as the cost of inspections
per fault detected, which indicates that ASA is a relatively affordable fault detection technique.
• The defect removal yield of ASA is not significantly different from that of inspections. The defect
removal yield of execution-based testing is two to three times higher than that of ASA and,
therefore, may be more effective at finding the defects remaining by that phase.
• The number of ASA faults in a module can be a fairly good measure of fault-prone module
identification prior to test.
• The mapping of ASI faults to ODC defect types indicated that ASI tools predominantly identify
two ODC defect types: Checking and Assignment.
• Approximately 90 percent of all the faults identified by manual inspection belong to Algorithm,
Documentation, and Checking faults.
• A large majority of test/customer-reported failures are in Function and Algorithm types.
• The 80-20 rule/Pareto effect found in faults and failures distribution analysis can be considered
as useful feedback to help us improve the code quality in future projects.
• A large percentage of programmer errors detected by ASA have the potential to cause security
vulnerabilities.
Jiang Zheng On the Value of Static Analysis for Fault Detection in
Software, IEEE, Apr. 2006

운용 결과 참고 II

More Question?
http://feeds.feedburner.com/Encarta
http://www.facebook.com/neohckim

http://creativecommons.org/licenses/by-sa/2.0/kr/