Do you want to make your systems more reliable and resilient before your organization becomes the next headline? View the slides from our recent webinar with Melinda Ballou, Program Director for IDC's Application Life-Cycle Management & Executive Strategies research. Melinda discusses the trends driving recent high-profile outages with increasing frequency, and gives practical advice on adapting your strategy for quality analysis and improving architectural design upfront. To view the recording, visit http://www.castsoftware.com/news-events/event/avoid-system-failure-idc?gad=ss

1. Webinar with Melinda Ballou
Program Director , IDC
Big Data, Big Problems:
Avoid System Failure
with Quality Analysis

2. CAST Confidential 1
Speakers
Melinda Ballou
Program Director, Application
Life-Cycle Management
IDC
Pete Pizzutillo
Director, Product Marketing
CAST

3. Sep-13© 2013 IDC 2
One Long Hot Week in August!!
One week
(from Aug 19 to Aug 26)
In the last
two weeks

4. Sep-13© 2013 IDC 3
“Process Gap”- High Cost of Inertia
Evolve Beyond Traditional ASQ to Better Address Risk
CloudMobile
Development Practices Complex SourcingLess Budget
Social Analytics

5. Sep-13© 2013 IDC 4
Industry Highlights: Disruptive Trends
 Diverse deployment demands for mobile, cloud, embedded drive corporate need for
architectural impact analysis for application portfolio, business dynamism is enabled by
software quality analysis -- & cost prohibitive
 Organizations re-invest, seeking to do more with fewer resources with financial and
staffing constraints; leveraging efficient approaches to restore and sustain high
performing, timely, business-critical software.
 Complex sourcing/off-shoring plus use of open source need strong teaming, effective
code management, testing, and metrics enabled by SQAM; Services driven environment
(SaaS/cloud, Devops emergence)
 Global economic competition and local compliance across geographies demand
quality, change and portfolio management, adaptability and rigor
 Flexible development paradigm with services creation increasingly drive technology
and business collaboration – strong agile emergence
 Emerging security issues (as driver) and virtualization/cloud (as enabling technology)
for SQAM adoption; ad hoc approaches unsustainable
 End-user experience and business impact challenges of rich Internet, mobile,
embedded, with social media collaboration/community opportunities
 Very public software failures increasing

6. Sep-13© 2013 IDC 5
“Quality Gap”- High Cost of Failure
Poor Quality = Increased Business Risk
Lost Revenue
($$$$$)
Lost Customers
Lost Productivity
Increased Costs
Lower ProfitsDamaged Brand

7. Sep-13© 2013 IDC 6
• Software Quality Analysis and Measurement: software tools that
enable organizations to observe, measure, and evaluate software
complexity, size, productivity, and risk (including technical &
structural quality, non-functional testing)
• Architectural assessment of design consequences (on software
performance, stability, adaptability, and maintainability)
• Static analysis and dynamic analysis
• Quality metrics for complexity, size, risk, and productivity to establish
baselines and to help judge project progress and resource capabilities
• Application portfolio evaluation through understanding the impact of
architectural flaws and dependencies
• In-phase prevention of additional software problems not easily
observable through typical ASQ tools.
SQAM Definition: Establishing a Strategy

8. Sep-13© 2013 IDC
Barriers to Traditional Testing –
SQAM Drivers
• Agile velocity demands immediate, frequent, iterative testing
• Lack of system resources constrains testing usage – expense limits
ability to mirror production configurations (mobile issues)
• Lack of architectural and design context for multi-modal deployment need
for management & coordination
• Challenges to test system configuration and impact to performance and
adaptability of design
• Lack of visibility into consequences of poor architecture with significant
impact to business or failed software
• Increasing occurrences of business critical failures are driving
engagement and interest in software quality analysis and measurement
7

9. Sep-13© 2013 IDC
ASQ Forecast with IDC Software
Quality Analysis & Mgmnt Segment
0
500
1,000
1,500
2,000
2,500
3,000
3,500
2009 2010 2011 2012 2013 2014 2015 2016 2017
$M
ASQ (6/13) SQAM (9/13 est)
19%
29%

10. Sep-13© 2013 IDC 9
IT and Business Challenges: Silos, Gaps
 Today’s applications are high-visibility, and carry a high cost-of-
failure -- customer self-serve, supplier/channel; key internal business
applications
 “Network effect” – failure in one leads to other failures
 The need for SQAM as part of quality life-cycle is key since G2000
organizations are split across groups:
– Business/users stakeholders
– Architects, Designers and Developers
– QA professionals
– Operational staff
 Must extend the Quality life-cycle across geographies, life cycle
phases and groups

11. Sep-13© 2013 IDC 10
Goals of Effective IT/Business
Alignment
New Business
Value
Reduced
Exposure
Innovation:
Maximize Upside
Through Technology-
Enabled Business
Processes
Compliance:
Minimize Downside
Through Risk
Management

12. Sep-13© 2013 IDC
CIO’s 2013 Personal Agenda
Q. In 2013, which of the following goals will be top of your personal agenda as CIO?
Please select your top 3 goals.
0% 5% 10% 15% 20% 25%
Implement a more rigorous process to evaluate
new ideas for IT to take on
Re-skill existing IT talent
More effectively attract new IT talent
Carve out more IT budget for new
projects/innovative projects
Focus IT organization on better understanding
the requirements of the consumers
Better align IT with the business
Foster a culture within IT where IT more often
provides a qualified "yes" to the business
Foster a culture within IT that drives more
innovation
Focus the IT organization more on business
strategy than technology strategy
US
WE
Total
n = 70
WE respondents = 21; US respondents = 49
Source: IDC 2013 CIO Agenda Survey, Fall 2012
5

13. Sep-13© 2013 IDC
By 2016, LOB executives will be directly
involved in 80% of new IT investments
It is Time to Revisit IT Planning, Quality
Governance and Portfolio Management Methods
Of the new internal IT projects initiated at your
company this year, what percentage will be led
under the following scenarios?
N = 57
Source: IDC 2013 CIO Sentiment
Survey, Fall 2012
Prediction
 58% of new IT investments in 2013 will
involve direct participation by LOB executives
 Companies will initiate an average of 40 new
IT projects in 2013 (with or without IT)
 Line of business’ participation in IT projects
will grow to 80% in 3 years
 The implications are vast on how the CIO
works with the line of business
Situation Assessment
 Cloud, social and mobile services are the
great equalizers, the balance-wheel of the
corporate machinery
 Notable instances of CEOs and CFOs driving
the migration to Cloud and Managed Services
8%
17%
33%
42%
0% 10% 20% 30% 40% 50%
Project solely led /
managed by the LOBs
Project led / managed by
LOBs, but subject to
review by IT
Project jointly led /
managed by IT and the
LOBs
Project solely led /
managed by IT
9

14. Sep-13© 2013 IDC
Three Key Challenges for IT
IT must deliver new applications that have greater business value
and higher quality, while managing costs …
in the face of these 3 key challenges:
 Increasing criticality of applications to the business
 Increasing complexity of software systems
– From web to mobile to embedded… encompassing social systems of
engagement to feed systems of record, performance demand with Big
Data Analytics for business optimization
 Increasingly distributed teams with multi-sourcing
– From onshore to offshore to open source
13

15. Sep-13© 2013 IDC
Source: IDC CloudTrack Survey, Winter 2012
n=493
Mean rating by
respondents
Q. How concerned are you about cloud...?
(1-5 scale; 5 = extremely concerned)
Security Tops Concerns: Risk Major Role

16. Sep-13© 2013 IDC 15
Coordinating across the Life-Cycle
• Coordinating architectural design, requirements, software analysis,
quality and operational performance is key across emerging technologies
• Failures and slow response time costs prohibitive for business areas
• Organizations should leverage quality automation through design,
requirements, unit test, system integration, pre-deployment & application
performance testing with emerging cloud / mobile /social platforms
• Evaluating software analysis with automation can help teams react and
manage user application experience
• As business requirements change, a cogent life-cycle approach enables
adaptive software analysis and responses
• Look to SQAM alternatives initially as an on-ramp to mobile, cloud and
multi modal dev – strategize through to deployment

17. Sep-13© 2013 IDC 16
IDC Calls to Action
• Across industries, poorly designed and problematic software leads
to brand perception impact above and beyond individual problems –
demand response
• The challenges of increased complexity and high-end development
across diverse platforms increase code problems, increase costs
and drive debilitating consequences resulting from defects pre- and
post-deployment
• Companies must become better educated about the business
consequences and labor costs of poor software design since
optimism masks the need for change
• Organizations should evaluate SQAM tools to supplement
traditional ASQ along with appropriate process and organizational
approaches

18. Sep-13© 2013 IDC 17
Summary
 Coordinate a Quality Life-Cycle approach that targets pragmatic
approaches to SQAM from design through to deployment to obtain
benefits
 Evaluate your organization’s current strategies for design,
application portfolio review, effective quality processes and
automated tools adoption
 Schisms between business, architects, development, testers and
operations must be addressed -- IT groups and the business must
build a common language, common metrics, and common tools and
practices that include SQAM
 Drive towards an effective quality strategy to help cut costs, increase
efficiency and business agility, to sustain brand, address competitive
challenges

19. Analyzing and Measuring Software Risks

20. CAST Confidential 19
Industry starting to pay attention to code quality
But code quality & hygiene, things traditional safe
guards identify are only a small part of the solution.
Sources: Li, et al. (2011). Characteristics of multiple component defects and
architectural hotspots: A large system case study. Empirical Software Engineering
“Tracking programming practices at the Unit Level alone may not translate into the
anticipated business impact,…most devastating defects can only be detected at the
System Level.”
8%
90%
Unit-Level
Flaws
System-
Level Flaws
% of
apps
defects
% of
repair
effort
92%
8%
52%
48% of downtime caused by
8% of system-level defects!

21. CAST Confidential
Business
Characteristic
Good Coding Practices
@ Unit-Level
Good Architectural Practices
@ Technology/System Levels
RELIABILITY Protecting state in multi-threaded environments
Safe use of inheritance and polymorphism
Resource bounds management, Complex code
Managing allocated resources, Timeouts
Multi-layer design compliance
Software manages data integrity and consistency
Exception handling through transactions
Class architecture compliance
PERFORMANCE
EFFICIENCY
Compliance with Object-Oriented best practices
Compliance with SQL best practices
Expensive computations in loops
Static connections versus connection pools
Compliance with garbage collection best practices
Appropriate interactions with expensive or remote resources
Data access performance and data management
Memory, network and disk space management
Centralized handling of client requests
Use of middle tier components vs. procedures/DB functions
SECURITY Use of hard-coded credentials
Buffer overflows
Missing initialization
Improper validation of array index
Improper locking
Uncontrolled format string
Input validation
SQL injection
Cross-site scripting
Failure to use vetted libraries or frameworks
Secure architecture design compliance
MAINTAINABILITY Unstructured and duplicated code
High cyclomatic complexity
Controlled level of dynamic coding
Over-parameterization of methods
Hard coding of literals
Excessive component size
Duplicated business logic
Compliance with initial architecture design
Strict hierarchy of calling between architectural layers
Excessive horizontal layers
Excessive multi-tier fan-in/fan-out
NUMBER OF ISSUES 90% of violations 10% of violations
BUSINESS IMPACT
52% of repair workload
10% of production downtime
48% of repair workload
90% of production downtime
Industry must focus on the flaws that matter

22. CAST Confidential
CAST Software Risk Prevention
 CAST solutions expose the weaknesses in complex multitier systems by identifying
the high severity engineering flaws undetectable by testing. CAST insures the
confidence that critical systems are free from vulnerabilities, either intentionally
designed into the software or accidentally inserted at anytime during its lifecycle.
1. Define the business-relevant software characteristics:
stability & resilience, performance efficiency, & security
important to your business.
2. Identify structural weaknesses and architectural hotspots
based on initial of applications.
3. Baseline and benchmark key risk indicators
against industry norms.
4. Monitor to ensure system do not degrade over time.
SOFTWARE RISK PREVENTION PROCESS
PEACE OF MIND - FROM THE INSIDE OUT.

23. CAST Confidential
Analysis strategy for typical IT application portfolio
22
Effort(ManDays/Year)
Importance to
Business
Highest Lowest
Critical Apps
Entire Application Portfolio
CAST AIP
 Deep Structural
Analysis
 Risk Detection
 Lean Application
Development
 Function Points &
Productivity
 Vendor Management
 Continuous
Improvement
CAST Highlight
 Fast Cloud-based
Delivery
 No source code
aggregation
 Key Metrics on Entire
Portfolio
 Size, Complexity and
Risk analytics
 Annual/Quarterly
Benchmark

24. CAST Confidential
Portfolio risk review with Highlight
23
QUICKLY SPOT SHORT TERM RISK – COMPLEX SYSTEMS LIKELY TO FAIL

25. CAST Confidential
ArchitectureCompliance
Enterprise IT applications require depth of analysis
24
 Intra-technology architecture
 Intra-layer dependencies
 Module complexity & cohesion
 Design & structure
 Inter-program invocation
 Security Vulnerabilities
Module Level
 Integration quality
 Architectural compliance
 Risk propagation
simulation
 Application security
 Resiliency checks
 Transaction integrity
 Function point & EFP
measurement
 Effort estimation
 Data access control
 SDK versioning
 Calibration across
technologies
System Level
Data FlowTransaction Risk
 Code style & layout
 Expression complexity
 Code documentation
 Class or program design
 Basic coding standards
Program Level
Propagation Risk
Java
EJB
PL/SQL
Oracle
SQL
Server
DB2
T/SQL
Hibernate
Spring
Struts
.NET
C#
VB
COBOL
C++
COBOL
Sybase IMS
Messaging
Java
Web
Services
1
2
3
JSP ASP.NETAPIs

26. CAST Confidential
CAST AIP - well beyond static analysis
Static Analysis
DependenciesCode
Pattern
Scanning
Data Flow Rule
Engine
Transaction
Finder Intelligent
Configuration
Content
Updater
The architectural assessment
of design consequences (on
software performance, stability,
adaptability, maintainability, and
security vulnerabilities) is an
area in which CAST excels and
successfully differentiates from
static analyzers.”
Architecture
Analysis
Behavioral
Simulation
Function
Points

27. CAST Confidential 26
Making risk management actionable
 Identify and stabilize are the tactical steps
 To harden and optimize is a move towards proactive
risk management that requires actionable processes
into the application lifecycle
 Quickly spot the riskiest applications
in your portfolio
 View overall Technical Quality
Risk Score
 View total number of critical
violations discovered.

28. CAST Confidential
• Upload Source Code
and documentation
• Complete a Technical
Survey
Application Assessment Process
TRANSFER
VALIDATE
INSIGHT
• CAST Consultant verifies
completeness of source code ,
artifacts, and technical survey.
• Verifies application boundaries.
• Results are published
to a private, secure portal
• Assessment report delivered
and presented to client
 Results by application
 Code Quality performance
 Benchmark across industry
Day 1 Day 2 – 4
Day 8
ANALYZE
• CAST Consultant performs the analysis.
• Using highly-sophisticated language
analyzers and more than 1000 industry-
best-practice rules, CAST assessment
identifies weakness in the application and
provides guidance on how to fix them.
• Verifies results with Client application
owner/SME
Day 4 – 7

29. CAST Confidential
Contact Information
Pete Pizzutillo
p.pizzutillo@castsoftware.com
www.castsoftware.com
blog.castsoftware.com
linkedin.com/company/cast
@OnQuality
slideshare.net/castsoftware