Secure application deployment in the age of continuous delivery

376 views

Published on

As presented at Open Source Open Standards (GovNet) (http://opensourceconference.co.uk/), this deck covers some of the material which operators of open source data centers and users of container and cloud technologies should be aware of when seeking to be security conscious.

Traditionally, when datacentre operators talk about application security, there has been a tendency to focus on issues related to key management, firewalls and data access. By contrast, application developers have a security focus which is more aligned with code analysis and fuzzing techniques. The reality is, secure application deployment principles extend from the infrastructure layer through the application and include how the application is deployed. With the prevalence of continuous deployment, it’s imperative to focus efforts on what attackers’ view as vulnerable; particularly in an environment where new exploits are being disclosed almost daily.

In this session we’ll present:

- How known vulnerabilities can make their way into production deployments
- How vulnerability impact is maximized
- A methodology for ensuring deployment of vulnerable code can be minimized
- A methodology to minimize the potential for vulnerable code to be redistributed

Published in: Government & Nonprofit
1 Comment
2 Likes
Statistics
Notes
  • Besides GRASSMARLIN what other open source OT tools are there for facility and infrastructure supervisory and control systems? https://www.wbdg.org/resources/cybersecurity.php
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
376
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
49
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • Image: http://morguefile.com/p/209940
  • http://www.istockphoto.com/photo/computer-crime-concept-gm516607038-89059287?st=9174601


    Source: http://www.verizonenterprise.com/verizon-insights-lab/dbir/2016/

    Every year since 2008, Verizon have published a report on the attempted data breaches occurring within their data centers. For 2015, they found close to 90% of them had either a financial or espionage component to them. This report is well worth the read, and there are a few key findings in this report we should all be aware of.

    Costs of data breaches are heavily skewed towards legal consultation and forensics, and not to the public components of credit monitoring or lawsuits
    Despite some vulnerabilities having been public for years, there remain vulnerable components in use
    Some of those components simply may not have a patch forthcoming for a variety of reasons.
  • Despite years of organizations spending energy protecting against attacks, it remains up to the attacker to define what’s valuable. Consider the case of ransomware. A police department in the town next to where I live was subjected to a raonsomeware attack. For roughly 500 USD in bitcoin, the attackers would decrypt the booking and evidence records they had just crypto locked. As an attacker, they likely had no knowledge of who they had attacked or what they had locked up. What mattered was the ransom, and that they had a police organization’s files didn’t factor into the equation.

  • https://www.cesg.gov.uk/guidance/open-source-software-%E2%80%93-exploring-risk-good-practice-guide-38
  • Let’s take a little bit of time and look at how an attack is created. Potential attackers have a number of tools at their disposal, and use a number of different tactics. In this case, the attacker wishes to create an attack on a given component. In order to be effective, they have two primary models. First they can actively contribute code in a highly active area of the component with an objective of planting a back door of some form. The hope being that their code will fail to be recognized as suspect given how quickly the area of code is evolving.

    Second they can look for areas of code which are stable, and the longer they’ve bene stable, the better. The reason for this is simple, old code is likely written by someone who isn’t with the project any longer, or perhaps doesn’t recall all assumptions present at the time the code was written. After all, its been long understood that even with the best developers, assumptions change and old code doesn’t keep up.

    The goal in both cases being to create an attack against the component, so they test, and fail, and iterate against the component until they’re successful or move on. Assuming they’re successful, they create a deployment tool and document the tool for others. Of course, given the publicity received by some recent vulnerabilities, a little PR goes a long way.

    Now there are responsible researchers who follow a similar workflow, and they legitimately attempt to work with component creators to disclose vulnerabilities. They too will publish results, but are less interested in creating the an attack beyond a proof of concept.




    http://www.istockphoto.com/photo/person-in-hooded-sweater-using-a-laptop-on-wooden-table-gm464503138-58544934?st=cf78f31
    http://www.istockphoto.com/photo/cloud-computing-gm518556682-90104967
  • https://www.cesg.gov.uk/guidance/open-source-software-%E2%80%93-exploring-risk-good-practice-guide-38

    If you’re using commercial software, the vendor is responsible for best practice deployment guidance, the notification of any security vulnerabilities and ultimately patches and workarounds for disclosed vulnerabilities. This is part of the deliverable they provide in return for their license fee.

    If you’re using open source software, that process becomes partly your responsibility. To illustrate the level of information you have to work with, let’s look at a media-wiki maintenance release from December 2015.

    “various special pages resulted in fata errors” – this clearly is something which needs resolution, but which pages? How do you test?
    “1.24.6 marks the end of support for 1.24.x” – this is good to know, but I hope it was published elsewhere.
    “However, 1.24.5 had issues (along with other versions) so it was thought fair to fix them” – This is a good thing, but can we expect this treatment in the future? From the title, we also have a fix for 1.23.x, but what other versions?


  • There is one thing we should all notice from this data: The vulnerable code was present for years until discovery. What may not be known, is that these vulnerabilities were found by researchers, not analysis tools.
  • Source: https://info.blackducksoftware.com/OpenSourceSA_LP.html

  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    On July 13 2015, the bug report associated with what would ultimately become CVE-2015-7547 was created.



    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html
    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch
  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    The report indicated that a traditional buffer management issue was present. Specifically it said “this change causes the thisanssizp pointer variable used in the recvfrom function on line 1282 to use the wrong size if a new buffer is created after the thisanssizp address has been changed at line 1257” and indicated that the result would be “The program will crash if the calculated size of the buffer used is 0. The recvfrom function will not crash, but any further accesses to the buffer where the bytes read was 0 from the recvfrom function will crash the program. ”

    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html
    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch
  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    On Feb 16, 2016, a CVE assignment was made to the bug list, and it further indicated the problem was introduced in May of 2008 in version 2.9.

    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html
    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch
  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    It wasn’t until two days later that the NVD (national vulnerability database) was updated to reflect the vulnerability. This meant that a staggered awareness situation ensued. When disclosures are staggered, there is an increased potential that a bad actor can take advantage of the situation. It’s important to note that this NVD entry came through US-CERT. One important thing to notice is that unlike the bug report, the overview contains significantly more actionable information (e.g. indicating that DNS is impacted “a crafted DNS response that triggers a call to the getaddrinfo function with the AF_UNSPEC or AF_INET6 address family, related to performing "dual A/AAAA DNS queries" and the libnss_dns.so.2 NSS module”


    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html
    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch

  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    Of course all most data center operators really want is to secure their environment, so the question really becomes “how to do that”. For an indication of how that plays out, we’ll look at VMware’s response as an example. The first thing most vendors do is create some form of security advisory, which in this case is “VMware Knowledge Base article 2144032”. In it they list the then current information about their exposure to the vulnerability. Often times, it’s minimal, but as they investigate, details are fleshed out. On February 22nd, they amended the KB article and created a security advisory “VMware Security Advisory VMSA-2016-0002” which included details on a patch for ESXi 5.5 and then updated a day later for a patch in ESXi 6.0. It’s not the least bit uncommon for some subordinate products, including older versions, to take longer to patch and on March 29th updates were announced to cover “vCenter Server Appliance (VCSA), 5.0 U3f, 5.1 U3c, and 5.5 U3c”

    Once you’ve a patch, then you can start working on resolving the issue in your infrastructure.

    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html
    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch

    https://www.youtube.com/watch?v=hkryI6eapOA

    http://blogs.vmware.com/security/2016/02/vmware-products-and-cve-2015-7547-glibc-getaddrinfo-security-issue.html
  • Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow

    From a timeline perspective, we’re looking at close to eight years from time of bug to vulnerability resolution. For much of that time, the impact was unknown. Starting in July of 2015, the scope of the issue started to be known, but close to seven months were required for investigation, triage, development of a fix and public disclosure to occur. Once the bug was known, the risk of exploitation increased slightly until that disclosure, but following the disclosure risk goes up dramatically. That’s one reason why knowing what’s running in your environment is so important. The overall goal being to reduce the time between disclosure and fix.



    https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html


    https://sourceware.org/bugzilla/show_bug.cgi?id=18665
    https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html

    https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT)

    http://cve.mitre.org/cve/cna.html

    https://openclipart.org/detail/200681/primary-patch

    https://www.youtube.com/watch?v=hkryI6eapOA
  • Source: Future of Open Source 2016 survey: https://www.blackducksoftware.com/2016-future-of-open-source
  • http://www.istockphoto.com/photo/strength-in-unity-gm514713440-88219133?st=af7fa36
  • Docker Container Security Scanner
    https://info.blackducksoftware.com/Security-Scan.html

    14 Day Free Trial to Black Duck Hub
    https://info.blackducksoftware.com/Demo.html

    Red Hat Atomic Host Integration (Requires Black Duck Hub)
    atomic scan --scanner blackduck [container]

  • Secure application deployment in the age of continuous delivery

    1. 1. Secure Application Deployment in the Age of Continuous Delivery OPENSOURCE: Open Standards
    2. 2. #whoami – Tim Mackey • Current roles: Senior Technical Evangelist; Occasional coder • Previously XenServer Community Manager • Cool things I’ve done • Designed laser communication systems • Early designer of retail self-checkout machines • Embedded special relativity algorithms into industrial control system • Find me • Twitter: @TimInTech ( https://twitter.com/TimInTech ) • SlideShare: slideshare.net/TimMackey • LinkedIn: www.linkedin.com/in/mackeytim
    3. 3. Security reality You can only protect what you know about. Defense in depth matters.
    4. 4. Attacks are big business In 2015, 89% of data breaches had a financial or espionage motive Source: Verizon 2016 Data Breach Report
    5. 5. Attackers decide what’s valuable … … and they have little fear.
    6. 6. EASY ACCESS TO SOURCE CODE Open source ubiquity makes it ready target OPEN SOURCE ISN’T MORE OR LESS SECURE THAN CLOSED SOURCE – ITS JUST EASIER TO ACCESS VULNERABILITIES ARE PUBLICIZED EXPLOITS ARE PUBLISHED
    7. 7. Anatomy of a new attack Potential Attack Iterate Test against platforms Document Don’t forget PR department Deploy
    8. 8. DEVELOPER DOWNLOADS OUTSOURCED DEVELOPMENT THIRD PARTY LIBRARIES CODE REUSE APPROVED COMPONENTS COMMERCIAL APPS OPEN SOURCE CODE Open source enters through many channels… …and vulnerabilities can come with it.
    9. 9. CLOSED SOURCE COMMERCIAL CODE • DEDICATED SECURITY RESEARCHERS • ALERTING AND NOTIFICATION INFRASTRUCTURE • REGULAR PATCH UPDATES • DEDICATED SUPPORT TEAM WITH SLA OPEN SOURCE CODE • “COMMUNITY”-BASED CODE ANALYSIS • MONITOR NEWSFEEDS YOURSELF • NO STANDARD PATCHING MECHANISM • ULTIMATELY, YOU ARE RESPONSIBLE Who is responsible for code and security?
    10. 10. 0 500 1000 1500 2000 2500 3000 3500 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Open Source Vulnerabilities Reported Per Year BDS-exclusive nvd Reference: Black Duck Software Knowledgebase, NVD Increasing number of OSS vulnerabilities
    11. 11. Automated tools miss most open source vulnerabilities Static & Dynamic Analysis Only discover common vulnerabilities 3,000+ disclosed in 2014 Less than 1% found by automated tools Undiscovered vulnerabilities are too complex and nuanced All possible security vulnerabilities
    12. 12. What do these all have in common? Heartbleed Shellshock GhostFreak Venom Since: Discovered: 2011 2014 1989 2014 1990’s 2015 2000 2015 2004 2015 Discovered by: Component: OpenSSL Riku, Antti, Matti, Mehta Bash Chazelas OpenSSL Beurdouche GNU C library Qualys researchers QEMU Geffner
    13. 13. Understand application contents Source: 2016 Open Source Security Report
    14. 14. Misaligned security investment
    15. 15. Distinct areas of risk • Open source license compliance • Ensure project dependencies are understood • Use of vulnerable open source components • Is component a fork or dependency? • How is component linked? • Operational risk • Can you differentiate between “stable” and “dead”? • Is there a significant change set in your future? • API versioning • Security response process for project
    16. 16. Total Quality Management Philosophies • Detect problems before product ships • Select components based on trust • Continuously identify issues and improve • Empower employees to solve problems • Implement the Deming Cycle • Plan for change and analyze risk • Do execute the plan in small steps • Check the results against the plan • Act on results to improve future outcomes • Manage with facts
    17. 17. Software development lifecycle Idea Spec Design Code Test Release
    18. 18. Software development lifecycle – threat model Idea Spec Design Code Test Release • As part of the specification and design, threat models are often created.
    19. 19. Software development lifecycle – static analysis Idea Spec Design Code Test Release • As part of the specification and design, threat models are often created. • During code creation and commits, static analysis is performed
    20. 20. Software development lifecycle – dynamic analysis Idea Spec Design Code Test Release • As part of the specification and design, threat models are often created. • During code creation and commits, static analysis is performed • Testing usually includes some form of dynamic testing
    21. 21. Traditional operations release process Deploy Measure ScaleMonitor Assess Release Update Spec
    22. 22. Oops – a vulnerability is disclosed – now what? DEVELOP SCM BUILD PACKAGE DEPLOY PRODUCTION BUG TRACKING REMEDIATE AND TRACK LICENSE COMPLIANCE AND SECURITY VULNERABILITIES FULL APP SEC VISIBILITY INTEGRATION BUILD / CI SERVER SCAN APPLICATIONS WITH EACH BUILD VIA CI INTEGRATION DELIVERY PIPELINE SCAN APPLICATIONS AND CONTAINERS BEFORE DELIVERY CONTINUOUS MONITORING OF VULNERABILITIES
    23. 23. Integrations matter …
    24. 24. Containers for application management
    25. 25. Knowledge is key. Can you keep up? glibc Bug Reported July 2015 Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    26. 26. Knowledge is key. Can you keep up? glibc Bug Reported July 2015 Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    27. 27. Knowledge is key. Can you keep up? glibc Vuln Introduced May 2008 glibc Bug Reported July 2015 CVE-2015- 7547 CVE Assigned Feb 16-2016 Low Security Risk Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    28. 28. Knowledge is key. Can you keep up? glibc Vuln Introduced May 2008 CVE-2015- 7547 CVE Assigned Feb 16-2016 glibc Bug Reported July 2015 National Vulnerability Database Vuln Published Feb 18-2016 Moderate Security Risk Low Security Risk Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    29. 29. Knowledge is key. Can you keep up? glibc Vuln Introduced National Vulnerability Database Vuln Published You Find It May 2008 CVE-2015- 7547 CVE Assigned Feb 16-2016 Feb 18-2016 glibc Bug Reported July 2015 Patches Available You Fix It Highest Security Risk Moderate Security Risk Low Security Risk Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    30. 30. Knowledge is key. Can you keep up? glibc Vuln Introduced National Vulnerability Database Vuln Published You Find It May 2008 CVE-2015- 7547 CVE Assigned Feb 16-2016 Feb 18-2016 glibc Bug Reported July 2015 Patches Available You Fix It Highest Security Risk Moderate Security Risk Low Security Risk Vuln: CVE-2015-7547: glibc getaddrinfo stack- based buffer overflow
    31. 31. Source: Future of Open Source 2016 Survey
    32. 32. A complete solution … Choose Open Source Proactively choose secure, supported open source SELECT Inventory Open Source Map Existing Vulnerabilities Maintain accurate list of open source components throughout the SDL Identify vulnerabilities during development VERIFY Track New Vulnerabilities Alert newly disclosed vulnerabilities in production MONITORREMEDIATE Fix Vulnerabilities Tell developers how to remediate
    33. 33. OVER TWO HUNDRED THIRTY E M P L O Y E E S 27USE BLACK DUCK SOFTWARE AWARD FOR INNOVATION GARTNER GROUP “COOL VENDOR” INNOVATIVE TECHNOLOGY OF THE YEAR - SECURITY 7 YEARS IN A ROW FOR SECURITY INNOVATION RANKED #38 OUT OF 500 SECURITY COMPANIES 7 YEARS IN A ROW 6 of the top 8 mobile handset vendors 7 of the top 10 SOFTWARE COMPANIES (44% OF TOP 100) 24 COUNTRIES 6 of the top 10 banks FORTUNE 100 Black Duck Created an Industry
    34. 34. 8,500 WEBSITES 350 BILLION LINES OF CODE 2,400 LICENSE TYPES 1.5 MILLION PROJECTS 76,000 VULNERABILITIES Comprehensive KnowledgeBase • Largest database of open source project information in the world. • Vulnerabilities coverage extended through partnership with Risk Based Security. • The KnowledgeBase is essential for identifying and solving open source issues.
    35. 35. We need your help Knowledge is power • Know what’s running and why • Define proactive vulnerability response process • Don’t let technology hype cycle dictate security Invest in defense in depth models • Don’t rely on perimeter security to do heavy lifting • Do look at hypervisor & container trends in security • Make developers and ops teams part of the solution • Do embed security into deployment process Together we can build a more secure data center
    36. 36. Free tools to help • Docker Container Security Scanner • https://info.blackducksoftware.com/Security-Scan.html • 14 Day Free Trial to Black Duck Hub • https://info.blackducksoftware.com/Demo.html • Red Hat Atomic Host Integration (Requires Black Duck Hub) • atomic scan --scanner blackduck [container]

    ×