Look Ma' No Hands - Automating Security the RightScale Way


Published on

RightScale Conference Santa Clara 2011: In the cloud, “manual” means “not doing it.” This session will provide guidance on how to automate portions of your security program using the RightScale Cloud Management Platform. Since we use RightScale to manage RightScale, most of the discussion will focus on what we are doing to accomplish the task of “automating security.” At the end of the session, you will have some very specific action items that you can take back to your environment to implement.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Poor application security leading to InjectionSQL injection was one of the top exploit in the Verizon Data Breach ReportPoor system configurations, leading to system compromisedNote the recent Windows RDP “exploit”. RDP left open, with Administrator having a well known password.Poor application configuration leading to application compromiseBrowsers that run scripts automaticallyPoor user habits leading to compromised credentials, that are then used to access dataUsers who click on attachments. Zeus bot, FakeAV, etc.
  • ConsiderationsTCP/UDP paths are notguaranteed!From source to destination (initial loads or updates)Across public networks or private?Once in the “cloud”Within Cloud Provider (CP) network where data is storedCrossing CP network where data is storedWithin the hypervisorCan someone:View or Modify it? Yes: Unencrypted, encrypted w/keysSo encrypt it , and protect the keysDeny it? Yes: packet manipulationNo way to prevent. Can use reliable transports and dedicated connections
  • Can someone:View or Modify it? Yes: Unencrypted, encrypted w/keysSo encrypt it , and protect the keysDeny it? Yes: local system access if improper ACL. Improper CP controlsProper ACL for local accounts.No way to prevent CP access. Risk assessment should be performed.
  • Can someone:View or Modify it? Yes: Memory is clearNeed to protect running memory from the InstanceNeed to trust the CPDeny it? No: Not specifically data. Can affect the instance, but really not practical to affect data in memory without affecting running instance stability
  • The questions are discussed in more detail in the next 2 slides
  • Trusted ImagesWindows w/ critical/recommend patch installed to image creation date Known configurationsServerTemplatesTrusted software repositoriesFrozen repositoriesScript the install and configRightScripts
  • Credentials are passed as launch parameters in a secure manner (usually SSL) to the cloud controller/provider
  • HowSame mechanism as in your enterpriseRightScale can be used to automate/orchestrate where needed, but does not do the patchingWindows: Windows Update, SUS, SCOM agent, etc.Think about application patchingLinux: Unfreeze repositories OR RightScript to update repository to latest testedLatter probably works better with Change Control Process
  • RightScale is using:Option: Unfreeze security repo, daily updates, use “app pinning”“apt-pinning” to prevent Collectd plugin to show current vulnsUnattended upgrade: Will not apply kernel reboot patches
  • Add Package: *Pin: release a=lucid-securityPin-Priority: 500Package: *Pin: release o=UbuntuPin-Priority: 50Why (and how) this works: The preferences file will pin all packages from ubuntu distribution to priority 50, which will make them less desirable than already installed packages. Files originating from security repository are given the default (500) priority so they are considered for installation. This means that only packages that are considered more desirable than currently installed ones are security updates. More information about pinning in the apt_preferencesmanpage.You can temporarily promote a certain distribution for updates with the --target-release option that works with apt-get and aptitude (at least) which will allow you pin certain releases so that they are eligible for upgrade.If you wish to use this for scripts only and not make it default for the system, you can place the rules in to some other location and use this instead: apt-get -o Dir::Etc::Preferences=/path/to/preferences_file upgradeThis will make apt look for the preferences file from a non-default location.The preferences file given as an example doesn't apply to third party repositories, if you wish to pin those too you can use apt-cache policy to easily determine the required keys for pinning.
  • Got it.. added another sed line to match /archive/<date> and change to /archive/latest. 20111013 could just as easily be added I guess..# CHANGE /MAJOR.MINOR FORMAT REPO URLS TO /MAJOR FORMAT, TO ALLOW ACCESS TO THE LATEST SECURITY UPDATES.sed -ri 's%centos/5.[0-9]%centos/5%' /etc/yum.repos.d/CentOS-*.repo
  • Look Ma' No Hands - Automating Security the RightScale Way

    1. 1. 1Look Ma’ No Hands:Automating Security the RightScale wayPhil CoxDirector, Security & Compliance, RightScale Watch the video of this presentation
    2. 2. 2#Agenda • Where are the risks? • Philosophy and musings • Pieces and Parts • Where RightScale shines • My Crystal BallReal Cloud Experience. Shared.
    3. 3. 3#Biggest real risks to data in the cloud?• The same things as when your data were not in the cloud. • Poor application security leading to Injection • Poor system configurations, leading to system compromised • Poor application configuration leading to application compromise • Poor user habits leading to compromised credentials, that are then used to access dataReal Cloud Experience. Shared.
    4. 4. 4#Common data exposure vectors in the cloud Data is typically exposed in the following three states: In Transit At Rest In ProcessReal Cloud Experience. Shared.
    5. 5. 5#We must protect data “In Transit”• Why? • You do not want the bad guys to see or modify your data • You can’t guarantee the path your data will take • You may have regulatory or contractual requirements to do so Map of Internet Traffic• Risk • Common Solutions • Sniffing along the path • Application Transport (SSL & TLS) • Modification of existing data • VPN (SSL, IPSEC, PPTP, L2TP) • Injection of new data • App level data encryption (custom)Real Cloud Experience. Shared.
    6. 6. 6#We must protect data “At Rest”• Why? Same as previous: You do not want unauthorized • Disclosure • Modification • Injection• Risks • Intrusion into Instance/Guest exposes data on its filesystem • Cloud provider access to ephemeral storage (e.g., EBS, SWIFT) • Cloud provider access to other storage options (e.g., S3, CloudFiles)• Common Solutions • Protection offered by running operating system (Access Control Lists) • *Encryption (and Key Management)* • SLA and Policies/Processes of the Cloud providerReal Cloud Experience. Shared.
    7. 7. 7#We must protect data while “In Process”• Why? Same as previous: You do not want unauthorized • Disclosure • Modification • Injection• Risk • Data is in clear in the memory of the Instance • Privileged users on a system can read memory • Hypervisor has access to instance memory• Common Solutions • Protect the system that is processing • Protect the hypervisor running the Instance • Limit administrative usersReal Cloud Experience. Shared.
    8. 8. 8#Philosophy and musings • Lets take "cloud" out of it for a moment • Just Good Enough Security • Figure out what “Secure” is for you • Best Practice is a red herring • Standard Practice is something to considerReal Cloud Experience. Shared.
    9. 9. 9#What is security automation? • “When I use a word, it means just what I choose it to mean- neither more nor less.” – Humpty Dumpty • So for our purposes today, automating security is about: • Building instances that meet “your” definition of security • Identifying vulnerabilities on running instances • Patching those vulnerabilitiesReal Cloud Experience. Shared.
    10. 10. 10#Some Compliance References • Baseline Requirements • HIPAA: 45 CFR 164.308(a)(4)* • ISO 27001: A.12.1.1, A.15.2.2 • PCI: 6.4 • NIST SP800-53: CM-2, SA-2, SA-4 • Vulnerability and Patch Management • HIPAA: 45 CFR 164.308 (a)(1)(i)(ii)(A) & (B), (5)(i)(ii)(B) • ISO 27001: A.12.5.1, A.12.5.2, A.12.6.1 • PCI: 2.2, 6.1, 6.2, 6.3.2, 6.4.5, 6.5.X, 6.6, 11.2 • NIST SP800-53: CM-3, CM-4, CP-10, RA-5, SA-7, SI-1, SI-2, SI-5Real Cloud Experience. Shared.
    11. 11. 11#Building instances that are secure • Starts with application design • You need to know what the systems will do, so you can build them accordingly • Think about: • What requirements for data in transit? • What requirements for data at rest? • What requirements for data in process? • What services will be exposed to untrusted parties? • What services will be exposed to trusted parties? • What services are only used internally?Real Cloud Experience. Shared.
    12. 12. 12#More on how design affects OpSec • What requirements for data in transit? • How do you handle the key material for SSL/TLS or data encryption? • Store it in on filesystem or in memory? • What requirements for data at rest? • Do you need runtime at reset security or off-line? • If in a database, will/can you use the database security or do you have to do it at the application? • If at the application layer, how do you manage keys? • What requirements for data in process? • Do you have to protect the data in memory/process? • This requires some HEAVY lifting and technology choicesReal Cloud Experience. Shared.
    13. 13. 13#More on how design affects OpSec • What services will be exposed to untrusted parties? • Will require diligence in patching and vulnerability management • What services will be exposed to trusted parties? • Likely less aggressive vulnerability management • Monitoring: Trust but verify? • What services are only used internally? • In reality will require less diligenceReal Cloud Experience. Shared.
    14. 14. 14#What you should have out of design • Services/Applications that will be run on what instances • OS types • Applications to be used • Network and applications Flows • Ports, Protocols, and Directions • Roles that are requiredReal Cloud Experience. Shared.
    15. 15. 15#Where RightScale shines • RightScale can be used to ensure that poor system and application configurations are not what cause you to lose your data • Use RightScale to: • Require data to be transmitted securely • Require data be stored securely • Ensure systems are appropriately patched and configured to minimize exposures • The core technologies are • RightImages • ServerTemplates • RightScripts • Repo’s and Mirrors • Security Motto: “Build it secure, keep it secure!”Real Cloud Experience. Shared.
    16. 16. 16#Build it SecureWhat Use Trusted Images Known Script the install Trusted Configurations and configuration Repository Start with Build with Modify with Build fromHow Multi-Cloud ServerTemplates RightScripts Frozen Repos ImagesReal Cloud Experience. Shared.
    17. 17. 17#Step 1: Standard images • RightImages are the only ones we can vouch for • Amazon has tons of available images, but we can’t vouch for them • Any RightScale Publisher would be a good choice • An ISV based image is likely OK, but we typically do not vet them • I’d work with professional services • In reality, you should start with ServerTemplates (next) as they will have selected vetted images alreadyReal Cloud Experience. Shared.
    18. 18. 18#Step 2: ServerTemplates • Dynamic configuration • Abstract role and behavior from cloud infrastructure • Predictable deployment • Cloud agnostic / portable • Object-oriented programming for sysadminsReal Cloud Experience. Shared.
    19. 19. 19#Step 2: ServerTemplates (con’t) Configuring servers Configuring servers with ServerTemplates: through bundling images: Custom MySQL 5.0.24 (CentOS Custom MySQL 5.2) (CentOS 5.0.24 5.4) MySQL 5.0.36 (CentOS 5.4) Setup DNS and IPs boot sequence MySQL 5.0.36 (Ubuntu 8.10) A setRestore last backup of configuration MySQL 5.0.36 (Ubuntu 8.10) 64bit directives that will install Frontend Apache 1.3 (Ubuntu 8.10) Configure MySQL and configure software on Frontend Apache 2.0 (Ubuntu 9.10) - top of the base image patched Install MySQL Server CMS v1.0 (CentOS 5.4) CMS v1.1 (CentOS 5.4) Install monitoring My ASP appserver (windows 2008) My ASP.net (windows 2008) – security update 1 Base Image My ASP.net (windows 2008) – security MultiCloudImage update 8 Very few and basic SharePoint v4 (windows 2003) – 32bit SharePoint v4 (windows 2003) –64bit CentOS Ubuntu SharePoint v4.5 (windows 2003) – Win 2003 5.2 CentOS 8.10 Ubuntu 64bit Win 2007 … 5.4 9.10Real Cloud Experience. Shared.
    20. 20. 20#ServerTemplates VS.• Integrated approach that puts together all the parts needed to architect single & multi-server deploymentsReal Cloud Experience. Shared.
    21. 21. 21#Step 2.x: RightScripts • RightScript is a mechanism to configure instances at boot time and to run additional scripts during the lifetime of an instance • A RightScript is an executable piece of code that can be run on a server • A RightScript consists of: • A script (typically written in Bash, Ruby, Perl, PowerShell, and now Chef) • A set of attachments that are downloaded from a storage location (e.g., S3) • A set of packages that are installed using the systems package manager • A set of input parameters that must be passed into the script • On ServerTemplates • Scripts or Recipe • /var/cache/rightscale/Real Cloud Experience. Shared.
    22. 22. 22#Important tangent: Logging and Auditing • Use ServerTemplates and RightScripts to integrate your logs into your enterprise SIEM • Look to a ISV’s or 3rd party SaaS SEM aggregator • Not for the faint of heart!Real Cloud Experience. Shared.
    23. 23. 23#Step 3: Identifying vulnerabilities • Out of scope of the RightScale core platform • Can “roll your own” or use ISV’s to help with this • Activities • Port and services scans • Validate implementation meets design • Nmap or typically included in Vulnerability scans • Vulnerability scans • SaaS services: CloudPassage*, SAINT, Rapid7, Qualys, Nessus, … • Build your own: SAINT, Rapid7, Qualys, Nessus, OpenVAS • Application testing • SaaS services are a good start: Whitehat, Vericode, HP, … • Manual testing is a must*: Whitehat, SystemExperts, Matasano, Aspect, … * Breaks the “automating” part of the talkReal Cloud Experience. Shared.
    24. 24. 24#Step 4: Patching • What • Update the Operating System • Update the applications • Validate the configuration • How • You can use the same mechanism as in your enterprise • *OR* • Use operational RightScripts to do it for you • *OR* • Use a partner ISV that specializes in that serviceReal Cloud Experience. Shared.
    25. 25. 25#Patching • Input form vulnerability management should drive this • Apply the security updates • Option 1: Apply to staging systems and run all your regression tests, then roll out • Option 2: Apply directly to production systems after a “cooling off period” • Option 3: Apply to a “canary” production system, wait 24 hrs, then apply en-masse • Option 4: Apply directly to production systems as soon as they are released • A couple points • Security patches are typically well tested before released • Applies well to Ubuntu, Windows, and RHEL • Not so well to CentOS • Upgrading the kernel is a bit touchier • pvgrub is your friendReal Cloud Experience. Shared.
    26. 26. 26#Ubuntu Security Patching • Ubuntu supports a security specific repo • Need to use RightScripts attached to ServerTemplates that points “security” repo to “latest” • Change the repost to point to “latest” • sed -i "s%ubuntu_daily/.* $(lsb_release -cs)-security%ubuntu_daily/latest $(lsb_release - cs)-security%" /etc/apt/sources.list.d/rightscale.sources.list • Update the list • “apt-get update” to Update the software list • Apply the updates • Pin what you don’t want to upgrade: /etc/apt/preferences.d/00rightscale • Upgrade what you do: apt-get upgrade • You need to decide if you want global updates or specific packages • https://help.ubuntu.com/community/AutomaticSecurityUpdate sReal Cloud Experience. Shared.
    27. 27. 27#CentOS Security Patching • CentOS does not have a security specific repo • Our CentOS /major repo now mirror the current • http://mirror.rightscale.com/centos/5/updates/i386/archive/<date> is a mirror of the /5.x (i.e. latest) repo on that day • Update repos to point to latest • Update /etc/yum.repos.d to point to the /major version • # Change /major.minor format Repo URLS to /major format • sed -ri s%centos/5.[0-9]%centos/5% /etc/yum.repos.d/CentOS-*.repo • # set latest or frozen date • sed -ri s/archive/[0-9]*/archive/latest/ /etc/yum.repos.d/CentOS-*.repo • sed -ri s/archive/([0-9]*|latest)/archive/20111013/g /etc/yum.repos.d/CentOS-*.repo • Update the list • yum check-update ( | grep updates) • Apply the updates (to specific packages)Real Cloud Experience. Shared.
    28. 28. 28#Security ISV’s to consider (alphabetical) • Centrify • Account controls integration with Active Directory • CloudPassage • Vulnerability management • Security event monitoring • Firewall management • TrendMicro • Secure data at restReal Cloud Experience. Shared.
    29. 29. 29#Recap • Design it properly • Build it to spec with RightImages, ServerTemplates, and RightScripts • Validate configurations and identify vulnerabilities with tools • Monitor with appropriate tools • Patch systems • ISV’s are your friend!Real Cloud Experience. Shared.
    30. 30. 30#Crystal Ball • Things that will help in the automation category • NIST Security Content Automation Protocol (SCAP) • Common Configuration Enumeration (CCE) • Common Platform Enumeration (CPE) • Common Vulnerabilities and Exposures (CVE) • Common Vulnerability Scoring System (CVSS) • Open Vulnerability and Assessment Language (OVAL) • Extensible Configuration Checklist Description Format (XCCDF) • CloudAudit • Policy and attestation • Me getting all this into the RightScale Platform Real Cloud Experience. Shared.
    31. 31. 31#My Info • phil@rightscale.com (securityguy@rightscale.com) • W – 805-243-0942 (x1142) • Skype: phil.cox.rs • Twitter: sec_prof • Linked-In: Philip CoxReal Cloud Experience. Shared.