Security Training: Necessary Evil, Waste of Time, or Genius Move?
Upcoming SlideShare
Loading in...5
×
 

Security Training: Necessary Evil, Waste of Time, or Genius Move?

on

  • 404 views

Most application risk managers agree that training software developers to understand security concepts can be an important part of any software security program. Couple that with the Payment Card ...

Most application risk managers agree that training software developers to understand security concepts can be an important part of any software security program. Couple that with the Payment Card Industry, who mandate that developers should have training in secure coding techniques as laid out in their Data Security Standard. Yet others call developer training "compliance-ware," a necessary evil and a tax on software development in the enterprise.
This presentation shares the results of a yearlong survey of nearly 1,000 software developers that captures their knowledge of application security before and after formal training. The survey queries developers from various backgrounds and industries, to better understand their exposure to secure development concepts and to capture a baseline for post-training improvements. The session also includes the results of a "retest" of a subset of respondents, to identify how much security knowledge they retained after a specific length of time. The results were surprising, and include information every application risk manager should know, particularly those who rely on training as part of an application security strategy.

Statistics

Views

Total Views
404
Views on SlideShare
366
Embed Views
38

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 38

http://www.denimgroup.com 38

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Security Training: Necessary Evil, Waste of Time, or Genius Move? Security Training: Necessary Evil, Waste of Time, or Genius Move? Presentation Transcript

  • © Copyright 2014 Denim Group - All Rights Reserved Security Training: Necessary Evil, Waste of Time or A Genius Move?
 " ! Research From Denim Group ! February 24, 2014! John B. Dickson, CISSP ! @johnbdickson !
  • © Copyright 2014 Denim Group - All Rights Reserved “I personally believe that training users in security is generally a waste of time, and that the money can be spent better elsewhere.” Bruce Schneier 2  
  • © Copyright 2014 Denim Group - All Rights Reserved ! • Both trying to change behaviors! –  Target audience has more power to say “no”! –  Deadlines and releases drive training! • For developers, infrequent, but more disruptive! –  15-45 minutes vs. 2-day class ! 3   How Developer Training is Different  
  • © Copyright 2014 Denim Group - All Rights Reserved Yet Training is Mandated ! •  PCI DSS 3.0 ü  Train developers in secure coding techniques, including how to avoid common coding vulnerabilities, and understanding how sensitive data is handled in memory ü  Testing Procedures: 6.5.a: Examine software development policies and procedures to verify that secure coding technique training is required for developers, based on best practices and guidance ü  Testing Procedures: 6.5.b: Interview a sample of developers to verify that they are knowledgeable in secure coding techniques ü  Testing Procedures: 6.5.c : Examine training records to verify that software developers received training on secure coding techniques, including how to avoid common coding vulnerabilities, and understanding how sensitive data is handled in memory 4  
  • © Copyright 2014 Denim Group - All Rights Reserved •  Harvard Business Review –  Large-scale organization development is rare –  Measurement of results is even rarer •  Workforce analytics rare –  More than 25% of survey respondents use little or no workforce analytics –  The vast majority (>61%) report their use as tactical, ad hoc, and disconnected from other key systems and processes 5   But Results Are Not Measured  
  • © Copyright 2014 Denim Group - All Rights Reserved •  Software development field growing 30% •  Turnover –  Industry – 14-15% –  General IT – ~20% –  Software Development – ~20 – 30% ! Sources: Bureau for Labor Statistics and Society of Human Resources Management! 6   Growth & Turnover Spur Sense of Urgency  
  • © Copyright 2014 Denim Group - All Rights Reserved Research Overview •  Focus: Assess the software developers depth of software security knowledge •  Purpose: To measure the impact of software security training on that level of understanding •  Survey size: 600 software developers surveyed in North America (US and Canada) •  Vertical markets represented: financial, government, retail, educational, technology, energy and healthcare segments 7  
  • © Copyright 2014 Denim Group - All Rights Reserved Respondent Demographics 24   23   148   53   56   128   0   20   40   60   80   100   120   140   160   #ofValidResponses" Company Size" 233   27   29   143   0   50   100   150   200   250   Software Developer! Quality Assurance! Architect! Other!#ofValidResponses" Primary Job Function" 8  
  • © Copyright 2014 Denim Group - All Rights Reserved Respondent Demographics Less than a Year" 10%" 1-2 Years" 8%" 2-4 Years" 12%" 4-7 Years" 11%" More than 7 Years" 59%" So#ware  Development  Experience   9  
  • © Copyright 2014 Denim Group - All Rights Reserved Respondent Demographics 168   86   56   27   95   0   20   40   60   80   100   120   140   160   180   None! Less than a Day! At least 1 day, but less than 2 days! At least 2 days, but less than 3 days! More than 3 days! #ofValidResponses" Previous App Sec Training" 10  
  • © Copyright 2014 Denim Group - All Rights Reserved §  15 Multiple Choice Quiz-Style Questions §  Targeted at Software Developers Ø  Varied by years of experience, amounts of previous training, primary job function, company industry and company size §  Distribution: Ø  Online (before and after) Ø  Hard-copy questionnaires given to instructor-led class trainees (before and after) Ø  Social media networks (sharing and some paid promotion with incentives) 11   Methodology  
  • © Copyright 2014 Denim Group - All Rights Reserved Hypotheses 1.  Most software developers do not have a basic understanding of software security concepts. 2.  Software security training can improve a developer’s knowledge of security concepts in the short-term. 3.  Certain industries, such as financial services, are more likely to have software developers that are already exposed to key software security concepts. 12  
  • © Copyright 2014 Denim Group - All Rights Reserved Sample Questions If  an  a6acker  were  able  to  view  sensi:ve  customer  records  they  should  not   have  had  access  to,  this  would  be  a(n)_______breach.       ___  Confiden3ality       ___  Integrity       ___  Availability       Authen:ca:on  is...       ___  Proving  to  an  applica3on  that  the  user  is  who  they  claim  to  be       ___  Confirming  that  the  user  is  allowed  to  access  a  certain  page  or  func3on       ___Verifying  that  the  data  displayed  on  a  given  page  is  authen3c       ___  Thoroughly  logging  all  of  a  user's  important  ac3vity     13  
  • © Copyright 2014 Denim Group - All Rights Reserved Sample Questions Marking  a  cookie  as  “secure”  will...       ___    Force  all  requests  that  use  the  cookie  to  use  SSL     ___    Prevent  an  aPacker  from  guessing  its  value     ___    Encrypt  it  when  sent  over  non-­‐SSL  requests     ___    Tell  the  browser  not  to  send  it  over  non-­‐SSL  requests     Which  of  the  following  will  help  protect  against  XSS?     ___    Only  accep3ng  URL  encoded  GET  parameters     ___    Not  using  any  JavaScript  in  the  applica3on     ___    Only  using  JavaScript  in  .js  files  stored  on  external  hosts     ___    Encoding  special  HTML  characters  in  data  as  it  is  rendered  to  the  page     14  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results Architects and software developers had a much higher level of knowledge than QA, yet in many organizations QA has a material role in application security 61%   56%   64%   56%   52%   54%   56%   58%   60%   62%   64%   66%   So_ware   Developer   Quality   Assurance   Architect   Other   Average  %  Correct   (Primary  Job  Func:on)   31%   22%   34%   18%   0%   5%   10%   15%   20%   25%   30%   35%   40%   So_ware   Developer   Quality   Assurance   Architect   Other   Group  Passing  Rate     (Primary  Job  Func:on)   15  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results Slightly more than half of the respondents correctly answered basic awareness questions on application but struggled with ways to operationalize appsec concepts 83%   69%   11%   0%   10%   20%   30%   40%   50%   60%   70%   80%   90%   #4: Cross Site Scripting (XSS) causes malicious scripts to execute on the user's… #7: Authentication is… #15: Which of the following will help protect against XSS? Percentage That Answered Correctly 16  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results •  Almost 100 percent could define input validation, demonstrating a choppy understanding of advanced secure coding knowledge •  Nearly 90 percent correctly identified proper session IDs which is reassuring 95%   88%   84%   86%   88%   90%   92%   94%   96%   #1:  Input  valida3on  is…   #11:  What  is  an  example  of  proper   session  IDs?   Percentage That Answered Correctly 17  
  • © Copyright 2014 Denim Group - All Rights Reserved 59%   74%   0%   10%   20%   30%   40%   50%   60%   70%   80%   Before  Training  (All)   A_er  Training  (All)   Average  %  correct   Key Survey Results •  Retention rose by more than 25 percent after completing secure coding training 18  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results Enterprises of more than 10,000 personnel had the lowest secure coding knowledge 61%   64%   58%   60%   62%   58%   55%   56%   57%   58%   59%   60%   61%   62%   63%   64%   65%   1-­‐24   Employees   25-­‐99   Employees   100-­‐499   Employees   500-­‐2499   Employees   2500-­‐9999   Employees   10,000  or   More   Employees   Average  %  Correct   (Company  Size)   33%   39%   26%   32%   32%   19%   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   1-­‐24   Employees   25-­‐99   Employees   100-­‐499   Employees   500-­‐2499   Employees   2500-­‐9999   Employees   10,000  or   More   Employees   Group  Passing  Rate     (Company  Size)   19  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results The majority of the respondents had no prior secure coding training, which might be surprising 168   86   56   27   95   0   20   40   60   80   100   120   140   160   180   None! Less than a Day! At least 1 day, but less than 2 days! At least 2 days, but less than 3 days! More than 3 days! #ofValidResponses" Previous App Sec Training" 20  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results There was no correlation between years of experience and knowledge of secure coding highlighting the continued need for effective security training 59%   60%   10%   20%   30%   40%   50%   60%   70%   80%   90%   100%   0  -­‐  7  years   More  than  7  years  experience   Average  %  Correct   Years  of  Development  Experience   Percentage  of  Correct  Answers   (Years  of  Development  Experience)   21  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results The respondents that had more than 3 days of app sec training in the past were able to answer more than half of the questions correctly 29%   15%   27%   22%   34%   0%   5%   10%   15%   20%   25%   30%   35%   40%   None   Less  than  a   Day   At  least  1   day,  but   less  than  2   days   At  least  2   days,  but   less  than  3   days   More  than   3  days   Percentage  of  group  who  correctly      answered  70%  or  more  ques:ons     Amount  of  Previous  Applica:on  Security  Training   Group  Passing  Rate     (Previous  App  Sec  Training)   59%   57%   60%   59%   63%   54%   55%   56%   57%   58%   59%   60%   61%   62%   63%   64%   None   Less  than  a   Day   At  least  1   day,  but  less   than  2  days   At  least  2   days,  but   less  than  3   days   More  than  3   days   Average  %  Score   Amount  of  Previous  Applica:on  Security  Training   Average  %  Correct   (Previous  App  Sec  Training)   22  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results 100% correctly identified where cross site scripting executes after completing training, an increase of almost 20 percentage points 83%   100%   0%   20%   40%   60%   80%   100%   120%   Before Training After Training Percentage With Correct Answers #4: Where Cross Site Scripting (XSS) Executes 23  
  • © Copyright 2014 Denim Group - All Rights Reserved Key Survey Results The number of respondents able to correctly identify what is application security more than doubled after training was complete 21%   55%   0%   10%   20%   30%   40%   50%   60%   Before Training After Training Correctly Identified Application Security Term 24  
  • © Copyright 2014 Denim Group - All Rights Reserved Software Developers Learn Differently than Companies “Teach” • Teaching methods are formalized and structured in order to be repeatable • Type of structures consist of: –  On-site & off-site classroom training –  E-learning for compliance –  Videos, webinars, etc. ! 25  
  • © Copyright 2014 Denim Group - All Rights Reserved Software Developers Learn Differently than Companies “Teach” • Teaching methods are formalized and structured in order to be repeatable • Type of structures consist of: –  On-site & off-site classroom training –  E-learning for compliance –  Videos, webinars, etc. ! 26  
  • © Copyright 2014 Denim Group - All Rights Reserved So How Do Developers Learn? •  Informally and in an unstructured way via:! •  Blogs & RSS feeds ! •  Social media with emphasis! •  Developer websites! •  Influential e-mail lists! •  Safarionline! 27  
  • © Copyright 2014 Denim Group - All Rights Reserved Don’t Ignore Basics of Training • Refresher training is still needed! • Training must be included in performance plans ! • Managers increasingly want an ROI! 28  
  • © Copyright 2014 Denim Group - All Rights Reserved Incentives Matter! ! 29  
  • © Copyright 2014 Denim Group - All Rights Reserved •  Software developers still largely do not understand key software security concepts •  73% of respondents “failed” the initial survey •  Average score of 59% before training •  However, software developers’ understanding of key software security concepts did increase after training •  QA staff struggled to understand software security concept vs. architects and software developers 30   CONCLUSION  
  • © Copyright 2014 Denim Group - All Rights Reserved Where do we Go from Here? 31  
  • © Copyright 2014 Denim Group - All Rights Reserved 
 
 " "Questions and Answers?" ! ! !John B. Dickson! ! !@johnbdickson! ! !john@denimgroup.com! 32