Successfully reported this slideshow.

Iterative Security: Secrets when you're not ready for Vault

1

Share

1 of 76
1 of 76

Iterative Security: Secrets when you're not ready for Vault

1

Share

Download to read offline

At CloudZero we're a startup and getting started is better than someone's definition of being perfect. And in security, getting started will have the most impact on fortifying your security posture and allow you to eventually reach that desired state.

There are great tools out there for managing secrets. But the problem space is still hard and the tools require time and investment. Let's apply an iterative approach to solving secrets management. Let's take an organization one step to the right from no secrets management to a state that a) improves their current state and b) sets them up a future improvement iteration.

related content the spawned this presentation:
* https://blog.threatstack.com/cloud-security-best-practicesfinding-securing-managing-secrets-part-1
* https://blog.threatstack.com/cloud-security-best-practices-finding-securing-managing-secrets-part-2

At CloudZero we're a startup and getting started is better than someone's definition of being perfect. And in security, getting started will have the most impact on fortifying your security posture and allow you to eventually reach that desired state.

There are great tools out there for managing secrets. But the problem space is still hard and the tools require time and investment. Let's apply an iterative approach to solving secrets management. Let's take an organization one step to the right from no secrets management to a state that a) improves their current state and b) sets them up a future improvement iteration.

related content the spawned this presentation:
* https://blog.threatstack.com/cloud-security-best-practicesfinding-securing-managing-secrets-part-1
* https://blog.threatstack.com/cloud-security-best-practices-finding-securing-managing-secrets-part-2

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Iterative Security: Secrets when you're not ready for Vault

  1. 1. tom@cloudzero.com @tmclaughbos Iterative Security:
 Secrets Management When You’re Not Ready For Vault tom@cloudzero.com @tmclaughbos
  2. 2. tom@cloudzero.com @tmclaughbos Who is this guy up here? His headshot is lasers and a cat
  3. 3. tom@cloudzero.com @tmclaughbos I’m Tom! • Community Engineer at CloudZero • Previously infrastructure engineer with a focus on automation.
  4. 4. tom@cloudzero.com @tmclaughbos Background & Biases Where I’m coming from
  5. 5. tom@cloudzero.com @tmclaughbos Background & Biases: I like startups
  6. 6. tom@cloudzero.com @tmclaughbos Background & Biases: I like startups
  7. 7. tom@cloudzero.com @tmclaughbos Background & Biases: Engineering is just a title
  8. 8. tom@cloudzero.com @tmclaughbos Does this look like you?
  9. 9. tom@cloudzero.com @tmclaughbos There is a lot of work to be done each day
  10. 10. tom@cloudzero.com @tmclaughbos For many of us, this ends up as reality…
  11. 11. tom@cloudzero.com @tmclaughbos We have all this technology! So why does these problems still exist?
  12. 12. tom@cloudzero.com @tmclaughbos How we present security
  13. 13. tom@cloudzero.com @tmclaughbos Security Paralysis I don’t know what to do
  14. 14. tom@cloudzero.com @tmclaughbos Security things you in ops might end up responsible for • Access controls • How much is too much access? • Password policies • How often should I force password rotation? • Wait, NIST has changed their recommendation? Don’t force rotation? • Patching • Do I patch immediately on every vendor release or test first? • and more…
  15. 15. tom@cloudzero.com @tmclaughbos
  16. 16. tom@cloudzero.com @tmclaughbos I am not a “security person”
  17. 17. tom@cloudzero.com @tmclaughbos Iterative Security Starting and progressively improving your security stance
  18. 18. tom@cloudzero.com @tmclaughbos Why is security hard for us?
  19. 19. tom@cloudzero.com @tmclaughbos OMG SECURITY!!!
  20. 20. tom@cloudzero.com @tmclaughbos Things we get excited about with security
  21. 21. tom@cloudzero.com @tmclaughbos Things we get excited about with security 0-Days!!!
  22. 22. tom@cloudzero.com @tmclaughbos Things we get excited about with security Hash collisions
  23. 23. tom@cloudzero.com @tmclaughbos Things we get excited about with security What the CIA/NSA/DoD has
  24. 24. tom@cloudzero.com @tmclaughbos Things we get excited about in security Logos
  25. 25. tom@cloudzero.com @tmclaughbos Things we don’t get excited about with security
  26. 26. tom@cloudzero.com @tmclaughbos Patching!
  27. 27. tom@cloudzero.com @tmclaughbos Not leaving MongoDB exposed to the internet with weak credentials
  28. 28. tom@cloudzero.com @tmclaughbos Not leaving Elasticsearch exposed to the internet with weak credentials
  29. 29. tom@cloudzero.com @tmclaughbos Learning from our mistakes
  30. 30. tom@cloudzero.com @tmclaughbos Many of us focus on the wrong things http://www.littlebobbycomic.com/projects/week-115/
  31. 31. tom@cloudzero.com @tmclaughbos How we present security
  32. 32. tom@cloudzero.com @tmclaughbos What we should be teaching • What are you trying to do? • Where do you start? • How do you progress
  33. 33. tom@cloudzero.com @tmclaughbos There’s a lot of info at the extremes
  34. 34. tom@cloudzero.com @tmclaughbos But this is where many of us are
  35. 35. tom@cloudzero.com @tmclaughbos We know not to… Put passwords, API keys, tokens, etc. in code.
  36. 36. tom@cloudzero.com @tmclaughbos But it still happens…
  37. 37. tom@cloudzero.com @tmclaughbos Where do you go from here?
  38. 38. tom@cloudzero.com @tmclaughbos Developing a threat model!
  39. 39. tom@cloudzero.com @tmclaughbos Be Realistic
  40. 40. tom@cloudzero.com @tmclaughbos USB sticks are a bigger threat than the man in the ceiling
  41. 41. tom@cloudzero.com @tmclaughbos A breach probably won’t end business
  42. 42. tom@cloudzero.com @tmclaughbos What are we trying to protect? • Intellectual property • Customer data (data about who our customers are) • Customer’s data (data from our customers) • etc.
  43. 43. tom@cloudzero.com @tmclaughbos What does our architecture look like?
  44. 44. tom@cloudzero.com @tmclaughbos Decompose the system
  45. 45. tom@cloudzero.com @tmclaughbos Decompose system: Perimeters
  46. 46. tom@cloudzero.com @tmclaughbos Decompose system: Data pipeline
  47. 47. tom@cloudzero.com @tmclaughbos Decompose system: Data pipeline
  48. 48. tom@cloudzero.com @tmclaughbos Identify threats • Exposed network ports (network) • Unpatched EC2 instances (host) • Weak secrets management(application) • User submitted data (application) • etc.
  49. 49. tom@cloudzero.com @tmclaughbos Document Threats • Weak password management • At two points in our infrastructure we’re not managing passwords • Both points involve highly valuable assets • Breach would be bad • Reputation loss -> customer loss • Data could be leveraged against our customers
  50. 50. tom@cloudzero.com @tmclaughbos Rate Threats Risk = Probability * Damage Potential
  51. 51. tom@cloudzero.com @tmclaughbos Rate Threats • D: Damage potential • R: Reproducibility • E: Exploitability • A: Affected users • D: Discoverability https://en.wikipedia.org/wiki/DREAD_(risk_assessment_model)
  52. 52. tom@cloudzero.com @tmclaughbos Rate Threats • D: Disaster if this is found (high) • R: Easy to reproduce (high) • E: Easy to exploit; requires existing access (medium) • A: Affects all users (high) • D: Not easy to find; users are hops away from issue (medium)
  53. 53. tom@cloudzero.com @tmclaughbos Putting a response into action Let’s manage those secrets
  54. 54. tom@cloudzero.com @tmclaughbos Constraints Time Complexity Risk
  55. 55. tom@cloudzero.com @tmclaughbos Constraints: Time
  56. 56. tom@cloudzero.com @tmclaughbos Constraints: complexity
  57. 57. tom@cloudzero.com @tmclaughbos Constraints: Risk of failure “It’s all code… We monitor it using Nagios.”
  58. 58. tom@cloudzero.com @tmclaughbos Constraints • Time: Few days to a few weeks • The faster we get this done the more likely we will finish. • Complexity: We’re going to go with what we know. • Less surprises. • Less to learn and get wrong. • Risk: Taking only as much risk as we’re ready for. • We’re moving fast! • Let’s limit the failure blast radius
  59. 59. tom@cloudzero.com @tmclaughbos Approaches to get you started Finally we secure some secrets
  60. 60. tom@cloudzero.com @tmclaughbos git-crypt • Encrypts secrets directly in your repository • Find secrets, rotate, and store them https://github.com/AGWA/git-crypt
  61. 61. tom@cloudzero.com @tmclaughbos git-crypt • Pro • You’ve done the exercise of auditing your code base • Cons • Symmetric encryption • Everyone needs the master password • TODO • Prevent key proliferation • Move to new secrets management when ready
  62. 62. tom@cloudzero.com @tmclaughbos Configuration Management: Puppet • Hiera-eyaml: Encrypt values in your Hiera hierarchy • Can use public key encryption • Multiple backends https://github.com/voxpupuli/hiera-eyaml
  63. 63. tom@cloudzero.com @tmclaughbos Configuration Management: Puppet • Pros • You’ve centralized your secrets in one repo. • Public key encryption support • Cons • May require manual intervention when rolling Puppetmasters. • May need to cleanup your Puppet code if you haven’t already moved to Hiera. • TODO: • Figure out master rekey strategy
  64. 64. tom@cloudzero.com @tmclaughbos Configuration Management: Ansible • Ansible Vault: Encrypts entire var files in playbook http://docs.ansible.com/ansible/playbooks_vault.html
  65. 65. tom@cloudzero.com @tmclaughbos Configuration Management: Ansible • Pros • You’ve done the exercise of auditing your code base • Cons • Symmetric encryption • Everyone needs the shared password • key proliferation • TODO • Preventing the proliferation of the Vault key • rekeying and rolling secrets.
  66. 66. tom@cloudzero.com @tmclaughbos S3 Buckets • Sneaker • Encrypt, store, and retrieve secrets from S3. https://github.com/codahale/sneaker
  67. 67. tom@cloudzero.com @tmclaughbos S3 Buckets • Pros • Secrets no longer live in repos • reduced secret proliferation • Secrets encrypted in S3. • Cons • How are you managing S3 buckets? • TODO • Manage your S3 buckets with CloudFormation, Terraform, etc. https://github.com/codahale/sneaker
  68. 68. tom@cloudzero.com @tmclaughbos What should we have gotten out of all this?
  69. 69. tom@cloudzero.com @tmclaughbos Less this…
  70. 70. tom@cloudzero.com @tmclaughbos …More this!
  71. 71. tom@cloudzero.com @tmclaughbos YOU CAN DO THIS!
  72. 72. tom@cloudzero.com @tmclaughbos Thank You! http://strayc.at/feedback
  73. 73. tom@cloudzero.com @tmclaughbos
  74. 74. tom@cloudzero.com @tmclaughbos Threat Modeling: startup edition https://twitter.com/CommitStrip/status/876830310780071936
  75. 75. tom@cloudzero.com @tmclaughbos Threat Modeling: startup edition response https://twitter.com/ErrataRob/status/876963608076439556
  76. 76. tom@cloudzero.com @tmclaughbos We know what not to do. We (think) we know where we want to be. But we don’t know how to get there.

Editor's Notes

  • Iterative Security
    Secrets Management When You’re Not Ready For Vault (or Conjur)

    I want to emphasize the Iterative Security part
    I changed the focus of this presentation from what’s on the agenda
    as I wrote tgis, I realized secrets management wasn’t the actual interesting part.

    This is partially based on past expereinces in my career
  • No new amazing tech in this presentation
    The technology isn’t the problem!!!
    We have good tech for managing secrets!
    Vault and Conjur exist.

    they’re really cool.
    they work well
  • I HAVE OTHER WORK THAT’S MORE IMPORTANT
    What’s more important?
    How about keeping the site up? Been there.

    Secrets management systems can be complicated systems to understand and operate
    They take time to understand
  • <SLIDE>

    laugh… But your friend next to you might be dealing with this
    A lot of us are or have been in situations we’re not proud of.
    We just don’t like to talk about this or admit it to others.
    I have been here previously in my career.
    Database passwords
    3rd party service tokens.
    etc.
  • This is how we present security to many people
    There is no in between
    There’s a lot of assumed knowledge from 1 to 2.
  • We don’t have a clue what to do.
    We don’t know how to get from point A to point B
    and given the choice of 8 other priorities
    we choose what we feel we can accomplish.
    We’re rewarded for what we accomplishment
    Security isn’t always one of our KPIs.
  • <SLIDE>
  • With all those questions

    <SLIDE>

    * We’ve put this in your hands and now it’s up to you to figure this out.
  • <SLIDE>
  • <SLIDE>
  • <SLIDE>

    This is a HUGE part of my role at Threat Stack as our Engineering Advocate
    Just listen
    find common problems
    address them
    secrets management that wasn’t as complicated and could be solved quick happened to be a topic.
    Posted this to reddit
    good response
  • going to discuss “iterative security”
    Just want to get folks started securing their environment
    How do we get people taking care of the basics and gradually improving?
    Wide open security groups?
    What about S3 buckets?
    we can start with lower hanging fruit and work up
  • <SLIDE>
  • OMG security! Let’s get all excited!
    SHINY OBJECTS!
  • What do people get excited about when talking security?
  • 0-days
    Nobody expects them!
    I couldn’t find a good 0-day vulnerability logo
    But nobody expects the Spanish Inquisition…
    Really, you just wake up and roll into work and, “WTH is this?”
  • Hash collisions (and other assorted cryptographic weaknesses)
    SHAttered: two different files that produce identical SHA1s
    Maybe someone will poison the Linux kernel repo
  • The government can hack things
    It considers The Cyber to be a battlefield and is doing what other states are doing.
  • * things with logos that everyone is talking about on Twitter/Slack/IRC
  • * What don’t we get excited about?
  • <SLIDE>
  • <SLIDE>
    Same damn thing a week or two later.
  • <SLIDE>

    MySQL was next after ES
    I did my part to warn the PostgreSQL community


    * This all took place in about 3 months
  • Some of us in here legitimately need to focus on
    0-days
    the logo of the month
    Many of us do not because we have not even done the basics
    we’re worried about APTs (advanced persistent threats) over our open databases.
  • Back to the owl
  • * What are you trying to look like?
    We talking headshot?
    side view
    frontal?
    Where do you start?
    How do you progress?
  • You can find something that works on your machine
    You can find something that works at Big McLarge Co that has an Ops team many times bigger than yours.
  • * Most of us are in the middle
    * There isn’t a lot out there on the left but just to the right of having gotten started solving our issues.
  • <SLIDE>
  • We’re going to apply the idea of iterating on security to Secrets Management
  • <SLIDE>
  • <SLIDE>
  • * Let’s solve this like a real problem at work.
  • * We’ll look at secrets management and impose some constraints on our decisions.
    In particular
    time
    How long will this work take?
    complexity
    How hard is this to do?
    How well can I understand it?
    failure risk
    What happens if the system fails?
    * These are normal business constraints we all face.
  • Going to take months
    You should do the perfect thing!
    I have other things that need to get done!


    for most of us, security isn’t the only thing on our plate.
    I have news systems to deploy, systems to tune, systems to fix, tools to build…
    We’re judged at work by that and not security.
  • Vault looks simple!

    How will you handle the master encryption key shards?
    Where they stored?
    Who has access?
    What storage backends meet your needs?
    How will you auth?

    * THANK YOU!
  • * “It’s all code… We monitor it using Nagios.”
  • Lets apply this!
    Secrets management if you’re not doing Vault (or Conjur)…
  • <slide>

    Build, satisfy needs, and throw away later

    You were at the worst point possible, you’ve at least taken a step…
  • <slide>

    reddit thread with author
  • Just like git-crypt
    <slide>
  • So, we’ll focus on less of this….
    Sweet car!
    I’m sure it’s fast
    But It’s also on fire.
  • We can manage this.
    roll in tomorrow and start putting out that garbage fiere
  • <slide>

    None of the solutions I talked about are Earth shattering.

    *One of you, go make things better tomorrow.
    Makes changes to prod on friday and leave
  • ×