Big Data, Big Problems: Avoid System Failure with Quality Analysis - Webinar with IDC analyst


Published on

Do you want to make your systems more reliable and resilient before your organization becomes the next headline? View the slides from our recent webinar with Melinda Ballou, Program Director for IDC's Application Life-Cycle Management & Executive Strategies research.

Melinda discusses the trends driving recent high-profile outages with increasing frequency, and gives practical advice on adapting your strategy for quality analysis and improving architectural design upfront. To view the recording, visit

Published in: Technology, Business
1 Comment
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Big Data, Big Problems: Avoid System Failure with Quality Analysis - Webinar with IDC analyst

  1. 1. Webinar with Melinda Ballou Program Director , IDC Big Data, Big Problems: Avoid System Failure with Quality Analysis
  2. 2. CAST Confidential 1 Speakers Melinda Ballou Program Director, Application Life-Cycle Management IDC Pete Pizzutillo Director, Product Marketing CAST
  3. 3. Sep-13© 2013 IDC 2 One Long Hot Week in August!! One week (from Aug 19 to Aug 26) In the last two weeks
  4. 4. Sep-13© 2013 IDC 3 “Process Gap”- High Cost of Inertia Evolve Beyond Traditional ASQ to Better Address Risk CloudMobile Development Practices Complex SourcingLess Budget Social Analytics
  5. 5. Sep-13© 2013 IDC 4 Industry Highlights: Disruptive Trends  Diverse deployment demands for mobile, cloud, embedded drive corporate need for architectural impact analysis for application portfolio, business dynamism is enabled by software quality analysis -- & cost prohibitive  Organizations re-invest, seeking to do more with fewer resources with financial and staffing constraints; leveraging efficient approaches to restore and sustain high performing, timely, business-critical software.  Complex sourcing/off-shoring plus use of open source need strong teaming, effective code management, testing, and metrics enabled by SQAM; Services driven environment (SaaS/cloud, Devops emergence)  Global economic competition and local compliance across geographies demand quality, change and portfolio management, adaptability and rigor  Flexible development paradigm with services creation increasingly drive technology and business collaboration – strong agile emergence  Emerging security issues (as driver) and virtualization/cloud (as enabling technology) for SQAM adoption; ad hoc approaches unsustainable  End-user experience and business impact challenges of rich Internet, mobile, embedded, with social media collaboration/community opportunities  Very public software failures increasing
  6. 6. Sep-13© 2013 IDC 5 “Quality Gap”- High Cost of Failure Poor Quality = Increased Business Risk Lost Revenue ($$$$$) Lost Customers Lost Productivity Increased Costs Lower ProfitsDamaged Brand
  7. 7. Sep-13© 2013 IDC 6 • Software Quality Analysis and Measurement: software tools that enable organizations to observe, measure, and evaluate software complexity, size, productivity, and risk (including technical & structural quality, non-functional testing) • Architectural assessment of design consequences (on software performance, stability, adaptability, and maintainability) • Static analysis and dynamic analysis • Quality metrics for complexity, size, risk, and productivity to establish baselines and to help judge project progress and resource capabilities • Application portfolio evaluation through understanding the impact of architectural flaws and dependencies • In-phase prevention of additional software problems not easily observable through typical ASQ tools. SQAM Definition: Establishing a Strategy
  8. 8. Sep-13© 2013 IDC Barriers to Traditional Testing – SQAM Drivers • Agile velocity demands immediate, frequent, iterative testing • Lack of system resources constrains testing usage – expense limits ability to mirror production configurations (mobile issues) • Lack of architectural and design context for multi-modal deployment need for management & coordination • Challenges to test system configuration and impact to performance and adaptability of design • Lack of visibility into consequences of poor architecture with significant impact to business or failed software • Increasing occurrences of business critical failures are driving engagement and interest in software quality analysis and measurement 7
  9. 9. Sep-13© 2013 IDC ASQ Forecast with IDC Software Quality Analysis & Mgmnt Segment 0 500 1,000 1,500 2,000 2,500 3,000 3,500 2009 2010 2011 2012 2013 2014 2015 2016 2017 $M ASQ (6/13) SQAM (9/13 est) 19% 29%
  10. 10. Sep-13© 2013 IDC 9 IT and Business Challenges: Silos, Gaps  Today’s applications are high-visibility, and carry a high cost-of- failure -- customer self-serve, supplier/channel; key internal business applications  “Network effect” – failure in one leads to other failures  The need for SQAM as part of quality life-cycle is key since G2000 organizations are split across groups: – Business/users stakeholders – Architects, Designers and Developers – QA professionals – Operational staff  Must extend the Quality life-cycle across geographies, life cycle phases and groups
  11. 11. Sep-13© 2013 IDC 10 Goals of Effective IT/Business Alignment New Business Value Reduced Exposure Innovation: Maximize Upside Through Technology- Enabled Business Processes Compliance: Minimize Downside Through Risk Management
  12. 12. Sep-13© 2013 IDC CIO’s 2013 Personal Agenda Q. In 2013, which of the following goals will be top of your personal agenda as CIO? Please select your top 3 goals. 0% 5% 10% 15% 20% 25% Implement a more rigorous process to evaluate new ideas for IT to take on Re-skill existing IT talent More effectively attract new IT talent Carve out more IT budget for new projects/innovative projects Focus IT organization on better understanding the requirements of the consumers Better align IT with the business Foster a culture within IT where IT more often provides a qualified "yes" to the business Foster a culture within IT that drives more innovation Focus the IT organization more on business strategy than technology strategy US WE Total n = 70 WE respondents = 21; US respondents = 49 Source: IDC 2013 CIO Agenda Survey, Fall 2012 5
  13. 13. Sep-13© 2013 IDC By 2016, LOB executives will be directly involved in 80% of new IT investments It is Time to Revisit IT Planning, Quality Governance and Portfolio Management Methods Of the new internal IT projects initiated at your company this year, what percentage will be led under the following scenarios? N = 57 Source: IDC 2013 CIO Sentiment Survey, Fall 2012 Prediction  58% of new IT investments in 2013 will involve direct participation by LOB executives  Companies will initiate an average of 40 new IT projects in 2013 (with or without IT)  Line of business’ participation in IT projects will grow to 80% in 3 years  The implications are vast on how the CIO works with the line of business Situation Assessment  Cloud, social and mobile services are the great equalizers, the balance-wheel of the corporate machinery  Notable instances of CEOs and CFOs driving the migration to Cloud and Managed Services 8% 17% 33% 42% 0% 10% 20% 30% 40% 50% Project solely led / managed by the LOBs Project led / managed by LOBs, but subject to review by IT Project jointly led / managed by IT and the LOBs Project solely led / managed by IT 9
  14. 14. Sep-13© 2013 IDC Three Key Challenges for IT IT must deliver new applications that have greater business value and higher quality, while managing costs … in the face of these 3 key challenges:  Increasing criticality of applications to the business  Increasing complexity of software systems – From web to mobile to embedded… encompassing social systems of engagement to feed systems of record, performance demand with Big Data Analytics for business optimization  Increasingly distributed teams with multi-sourcing – From onshore to offshore to open source 13
  15. 15. Sep-13© 2013 IDC Source: IDC CloudTrack Survey, Winter 2012 n=493 Mean rating by respondents Q. How concerned are you about cloud...? (1-5 scale; 5 = extremely concerned) Security Tops Concerns: Risk Major Role
  16. 16. Sep-13© 2013 IDC 15 Coordinating across the Life-Cycle • Coordinating architectural design, requirements, software analysis, quality and operational performance is key across emerging technologies • Failures and slow response time costs prohibitive for business areas • Organizations should leverage quality automation through design, requirements, unit test, system integration, pre-deployment & application performance testing with emerging cloud / mobile /social platforms • Evaluating software analysis with automation can help teams react and manage user application experience • As business requirements change, a cogent life-cycle approach enables adaptive software analysis and responses • Look to SQAM alternatives initially as an on-ramp to mobile, cloud and multi modal dev – strategize through to deployment
  17. 17. Sep-13© 2013 IDC 16 IDC Calls to Action • Across industries, poorly designed and problematic software leads to brand perception impact above and beyond individual problems – demand response • The challenges of increased complexity and high-end development across diverse platforms increase code problems, increase costs and drive debilitating consequences resulting from defects pre- and post-deployment • Companies must become better educated about the business consequences and labor costs of poor software design since optimism masks the need for change • Organizations should evaluate SQAM tools to supplement traditional ASQ along with appropriate process and organizational approaches
  18. 18. Sep-13© 2013 IDC 17 Summary  Coordinate a Quality Life-Cycle approach that targets pragmatic approaches to SQAM from design through to deployment to obtain benefits  Evaluate your organization’s current strategies for design, application portfolio review, effective quality processes and automated tools adoption  Schisms between business, architects, development, testers and operations must be addressed -- IT groups and the business must build a common language, common metrics, and common tools and practices that include SQAM  Drive towards an effective quality strategy to help cut costs, increase efficiency and business agility, to sustain brand, address competitive challenges
  19. 19. Analyzing and Measuring Software Risks
  20. 20. CAST Confidential 19 Industry starting to pay attention to code quality But code quality & hygiene, things traditional safe guards identify are only a small part of the solution. Sources: Li, et al. (2011). Characteristics of multiple component defects and architectural hotspots: A large system case study. Empirical Software Engineering “Tracking programming practices at the Unit Level alone may not translate into the anticipated business impact,…most devastating defects can only be detected at the System Level.” 8% 90% Unit-Level Flaws System- Level Flaws % of apps defects % of repair effort 92% 8% 52% 48% of downtime caused by 8% of system-level defects!
  21. 21. CAST Confidential Business Characteristic Good Coding Practices @ Unit-Level Good Architectural Practices @ Technology/System Levels RELIABILITY Protecting state in multi-threaded environments Safe use of inheritance and polymorphism Resource bounds management, Complex code Managing allocated resources, Timeouts Multi-layer design compliance Software manages data integrity and consistency Exception handling through transactions Class architecture compliance PERFORMANCE EFFICIENCY Compliance with Object-Oriented best practices Compliance with SQL best practices Expensive computations in loops Static connections versus connection pools Compliance with garbage collection best practices Appropriate interactions with expensive or remote resources Data access performance and data management Memory, network and disk space management Centralized handling of client requests Use of middle tier components vs. procedures/DB functions SECURITY Use of hard-coded credentials Buffer overflows Missing initialization Improper validation of array index Improper locking Uncontrolled format string Input validation SQL injection Cross-site scripting Failure to use vetted libraries or frameworks Secure architecture design compliance MAINTAINABILITY Unstructured and duplicated code High cyclomatic complexity Controlled level of dynamic coding Over-parameterization of methods Hard coding of literals Excessive component size Duplicated business logic Compliance with initial architecture design Strict hierarchy of calling between architectural layers Excessive horizontal layers Excessive multi-tier fan-in/fan-out NUMBER OF ISSUES 90% of violations 10% of violations BUSINESS IMPACT 52% of repair workload 10% of production downtime 48% of repair workload 90% of production downtime Industry must focus on the flaws that matter
  22. 22. CAST Confidential CAST Software Risk Prevention  CAST solutions expose the weaknesses in complex multitier systems by identifying the high severity engineering flaws undetectable by testing. CAST insures the confidence that critical systems are free from vulnerabilities, either intentionally designed into the software or accidentally inserted at anytime during its lifecycle. 1. Define the business-relevant software characteristics: stability & resilience, performance efficiency, & security important to your business. 2. Identify structural weaknesses and architectural hotspots based on initial of applications. 3. Baseline and benchmark key risk indicators against industry norms. 4. Monitor to ensure system do not degrade over time. SOFTWARE RISK PREVENTION PROCESS PEACE OF MIND - FROM THE INSIDE OUT.
  23. 23. CAST Confidential Analysis strategy for typical IT application portfolio 22 Effort(ManDays/Year) Importance to Business Highest Lowest Critical Apps Entire Application Portfolio CAST AIP  Deep Structural Analysis  Risk Detection  Lean Application Development  Function Points & Productivity  Vendor Management  Continuous Improvement CAST Highlight  Fast Cloud-based Delivery  No source code aggregation  Key Metrics on Entire Portfolio  Size, Complexity and Risk analytics  Annual/Quarterly Benchmark
  24. 24. CAST Confidential Portfolio risk review with Highlight 23 QUICKLY SPOT SHORT TERM RISK – COMPLEX SYSTEMS LIKELY TO FAIL
  25. 25. CAST Confidential ArchitectureCompliance Enterprise IT applications require depth of analysis 24  Intra-technology architecture  Intra-layer dependencies  Module complexity & cohesion  Design & structure  Inter-program invocation  Security Vulnerabilities Module Level  Integration quality  Architectural compliance  Risk propagation simulation  Application security  Resiliency checks  Transaction integrity  Function point & EFP measurement  Effort estimation  Data access control  SDK versioning  Calibration across technologies System Level Data FlowTransaction Risk  Code style & layout  Expression complexity  Code documentation  Class or program design  Basic coding standards Program Level Propagation Risk Java EJB PL/SQL Oracle SQL Server DB2 T/SQL Hibernate Spring Struts .NET C# VB COBOL C++ COBOL Sybase IMS Messaging Java Web Services 1 2 3 JSP ASP.NETAPIs
  26. 26. CAST Confidential CAST AIP - well beyond static analysis Static Analysis DependenciesCode Pattern Scanning Data Flow Rule Engine Transaction Finder Intelligent Configuration Content Updater The architectural assessment of design consequences (on software performance, stability, adaptability, maintainability, and security vulnerabilities) is an area in which CAST excels and successfully differentiates from static analyzers.” Architecture Analysis Behavioral Simulation Function Points
  27. 27. CAST Confidential 26 Making risk management actionable  Identify and stabilize are the tactical steps  To harden and optimize is a move towards proactive risk management that requires actionable processes into the application lifecycle  Quickly spot the riskiest applications in your portfolio  View overall Technical Quality Risk Score  View total number of critical violations discovered.
  28. 28. CAST Confidential • Upload Source Code and documentation • Complete a Technical Survey Application Assessment Process TRANSFER VALIDATE INSIGHT • CAST Consultant verifies completeness of source code , artifacts, and technical survey. • Verifies application boundaries. • Results are published to a private, secure portal • Assessment report delivered and presented to client  Results by application  Code Quality performance  Benchmark across industry Day 1 Day 2 – 4 Day 8 ANALYZE • CAST Consultant performs the analysis. • Using highly-sophisticated language analyzers and more than 1000 industry- best-practice rules, CAST assessment identifies weakness in the application and provides guidance on how to fix them. • Verifies results with Client application owner/SME Day 4 – 7
  29. 29. CAST Confidential Contact Information Pete Pizzutillo @OnQuality