Successfully reported this slideshow.
Your SlideShare is downloading. ×

Symfony finally swiped right on envvars

Symfony finally swiped right on envvars

A presentation I gave on September 26 at the Melbourne Symfony developers group on using Environment Variables (envvars) in Symfony and managing secrets in your PHP applications.

For more information on these subjects, check out the supporting piece I wrote: https://samjarrett.com.au/swipe-right

A presentation I gave on September 26 at the Melbourne Symfony developers group on using Environment Variables (envvars) in Symfony and managing secrets in your PHP applications.

For more information on these subjects, check out the supporting piece I wrote: https://samjarrett.com.au/swipe-right

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Symfony finally swiped right on envvars

  1. 1. Or,“why do we keep baking secrets into our artifacts?” Symfony finally swiped right on envvars
  2. 2. Or,“why do we keep baking secrets into our artifacts?” Symfony finally swiped right on envvars just a collection of cool gifs
  3. 3. Hi, I’m Sam ● Long time listener, first time caller. ● Co-organiser of this group. ● Using Symfony since 2.0 beta. ● Day job - ops @ REA. ● Docker & build automation fanboy. ● Have committed terrible atrocities with VCS and build systems in the past.
  4. 4. Symfony is not your grandma’s framework unique as a web framework
  5. 5. Most frameworks re-read config at boot. (Every. page. load.)
  6. 6. $ bin/elasticsearch ...waits 30 seconds... $ curl localhost:9200 [timeout] ...waits 30 more seconds... $ curl localhost:9200 [timeout]
  7. 7. Symfony compiles the DI container to PHP, including all config values.
  8. 8. Let’s talk about secrets, baby.
  9. 9. What is a secret?
  10. 10. ● Database credentials ● SMTP creds ● Payment gateway keys ● AWS credentials ● SMS tokens ● EDM service API keys Beyond 11 herbs and spices
  11. 11. The modern app has so many.. ● S3 bucket names ● SSH or deploy keys ● JWT keys or certificates ● Service n’s API key ● Other internal system hostnames But really, an application secret is anything you wouldn’t put directly in the HTML markup itself.
  12. 12. Here’s why we get a bad rep as php developers[1][2] … 1: I’ve seen all of these examples used 2: you probably have, too
  13. 13. <?php class Config { const LIVE_DATABASE_STRING = 'mysql://root:root@localhost:3306/pro const TEST_DATABASE_STRING = 'mysql://root:root@localhost:3306/tes public static function getDatabaseString() { // @TODO add magic to determine environment return self::LIVE_DATABASE_STRING; } // ...etc The know it all “pattern”
  14. 14. The build it & they’ll come “pattern” 1. put your prod & test values in a config file 2. run “make config” or some other magic build script 3. generate PHP from template: <?php class Config { const DATABASE_STRING = '{{ database_string }}'; public static function getDatabaseString() { return self::DATABASE_STRING;
  15. 15. dropping a prod database on a misconfigured test site. - my biggest professional fuck up. probably?
  16. 16. config_dev.yml: doctrine: dbal: url: 'mysql://root:root@localhost:3306/dev' The config.yml is the new Config class…. “pattern” config_prod.yml: doctrine: dbal: url: 'mysql://root:fk!ash#@localhost:3306/pr
  17. 17. Into today*... * as is common in many Symfony projects
  18. 18. Note: This is the best of the bad, but Symfony makes it insecure & hard to update. 1. define your config keys in a file called parameters.yml.dist, set sane defaults 2. store the actual configuration on S3 / build servers / CI / somewhere else 3. at build-time, copy the file in & build the container (cache-warm) The Fabien told me to “pattern”
  19. 19. OK. What’s wrong with this? How secure is your CI? Who has access to wherever the parameter values are stored?
  20. 20. OK. What’s wrong with this? Rolling the keys means re-deploying the application.
  21. 21. OK. What’s wrong with this? Secrets are written into the compiled Container class file. Try it yourself: $ grep “database.host” var/cache/prod/*.php
  22. 22. OK. What’s wrong with this? Creating new deployments is hard.
  23. 23. OK. What’s wrong with this? Production artifact differs to test.
  24. 24. OK, so what’s the answer?
  25. 25. Introducing environment variables ✅ Easy to set up in the app ✅ Functions just like another parameter ✅ Not compiled into the container ✅ Like other parameters, allows default values to be set ✅ Updating values is as simple as updating wherever it’s stored & reload PHP
  26. 26. Introducing environment variables ❌ Can’t access outside services - i.e. controllers etc - may require app refactoring ❌ Can be difficult to implement if you run your app outside of Docker ❌ Some libraries don’t quite support envvars yet (there’s a workaround though!)
  27. 27. Porting an existing app is easy.
  28. 28. parameters: env(DATABASE_URL): mysql://root:root@db:3306/symfony?charset= env(MAILER_HOST): smtp.sparkpostmail.com env(MAILER_PORT): 587 env(MAILER_USER): SMTP_Injection env(MAILER_PASS): env(SECRET): ThisTokenIsNotSoSecretChangeIt short_link_host: %env(SHORT_LINK_HOST)% env(SHORT_LINK_HOST): fallback.local Default values, get overridden if provided in the env How I dealt with libraries that don’t support env vars or values needed in controllers Note the structure around the name
  29. 29. doctrine: dbal: url: '%env(DATABASE_URL)%' orm: auto_generate_proxy_classes: '%kernel.debug%' naming_strategy: doctrine.orm.naming_strategy.underscore auto_mapping: true swiftmailer: transport: "smtp" host: "%env(MAILER_HOST)%" port: "%env(MAILER_PORT)%" username: "%env(MAILER_USER)%" password: "%env(MAILER_PASS)%" spool: { type: "memory" } How they’re used - in configuration YAML files...
  30. 30. services: amazon_s3: class: AwsS3S3Client arguments: - credentials: key: "%env(AWS_ACCESS_KEY_ID)%" secret: "%env(AWS_ACCESS_SECRET_KEY)%" region: "%env(AWS_REGION)%" version: "2006-03-01" How they’re used - in service configuration files...
  31. 31. While you’re porting ● Consider what values actually change between deployment environments ● Anything that doesn’t change, should go into config.yml, config_dev.yml or config_prod.yml (a la current standard edition practice). ● Anything that changes is a good candidate to live in the environment.
  32. 32. Any gotchas? ● Need to defer making decisions on anything that changes per-deployment until later. i.e. any decisions made in a container extension. ● Dynamic parameters don’t solve this %param% => %env(PARAM)% ● If it’s your bundle/extension, you can use factory services to work around this.
  33. 33. The Dotenv component in 3.3 ● Symfony flex uses it out of the box! ● A small class that is added in your app/app_dev.php and console scripts ● Provides env-var like support in places you can’t easily set variables (dev!) ● In Flex, it replaces the parameters.yml paradigm ● Relatively easy to back-port into the full-stack framework
  34. 34. Or…
  35. 35. You can use docker in development!
  36. 36. It’s not (all) about security.
  37. 37. Environment vars are not perfect ● Does not solve security for anyone with access to the machine. ● Can still be accessed from outside your app - i.e. anything in shell_exec() calls. ● The PHP user or someone on CLI can expose to another program. ● Any insecure libraries in your app can access it, too. ● Can unintentionally turn up in logs, the Symfony profiler, etc. ● The plain-text values are probably still stored somewhere.
  38. 38. How do I make it better? ● Current PR: github/symfony/symfony#23901 (due 3.4) adds features to support docker/kubernetes secrets & make it extendable. ● Once it’s available to us, it might be like this… ○ on AWS? KMS-encrypted envvars or SSM Param. Store. ○ Not AWS? G-Cloud KMS, or other open-source, e.g. Hashicorp Vault, Pinterest Knox, Lyft Confidant. ● In the meantime: do you host on AWS? Use KMS-encrypted environment variables or fetch from SSM Param. Store at container boot.
  39. 39. ● Any of these principles a surprise? Read about 12-factor apps ● Segment have a great write-up on managing secrets on AWS on their blog. ● KMS-ed envvars (enc. strings): REA’s Shush is a tiny go library to help. ● SSM param. store (central management): Segment’s Chamber can help. ● Taking inspo from k8s, Docker swarm mode now has secrets built-in. ● Not using Docker? Can be quite complicated to set env across multiple users + daemons. Apache, and CLI users share a syntax, PHP-FPM is different. ● Example of porting a Symfony app to Docker - for the adventurous. Further reading & tools
  40. 40. thank you. stalky stalk: @sammyjarrett or linkedin.com/in/samjarrett more detail and these slides published at samjarrett.com.au/swipe-right

Editor's Notes

  • Since 2.0 beta, before all the current patterns we now use existed, before the documentation was any good.
    Terrible atrocities: here to atone, really.
  • For PHP apps, booting refers to the app loading on each page load
    Not just something that occurs in PHP. Ruby, Python, Node, etc all do this.
    In comparison, Symfony demands that you compile the application before you boot it. We call it building the container. Or clearing the cache.
    Symfony steals a lot from Java, which also also suffers from the same issue, but you notice it differently...
    Here’s something I’m sure we’re all familiar with...
  • This is familiar with any time you’ve ever booted a java app. Building the configuration tree and configuring the services is mostly why they take so long to start accepting traffic!
  • Which gives us a good speed improvement because the app doesn’t have to load all of it’s configuration and build the container at runtime!
    This creates a big problem when we have complex build and deploy processes that need to operate at scale (how do you deploy to 20+ instances quickly? prebuild!) and still protect secret values
    Modern deployment practices emphasise keeping the artifact static between environments, to be able to more easily replicate issues back in development or testing environments.
    Container compilation means that we can’t guarantee that our artifact compiles correctly or that it is exactly the same in production. So, how do we solve this in symfony?
  • But this naturally creates issues when we have secrets in our apps, and yep, the modern app has a lot of them
  • What is a secret?
    Secrets can be any form of data that you wouldn’t want to be widely spread around.
  • Internal hostnames (i.e. unprotected internal systems you’re hiding): redis, memcached, finance systems, etc
    Lead developer’s email address (for emailing emerging issues, I guess)
    Pagerduty/OpsGenie alert topics/email address (don’t want these abused overnight waking up on call staff!)
    Slack tokens

    Summary: secret is anytthing you wouldn’t want to put in the markup
  • Create a single class or factory class, and delegate responsibility for it to provide you with answers,
    Ultimately embedding the secret for all environments in it.
    Compromise here on either test or production means all of your secrets are gone.
  • Use ini, json, yaml, java.properties style or some other configuration format to specify all of your values
    Create a php-class as a template full of placeholders that a template parser can quickly generate the final value for you
    Generate the PHP from the template before deploying (as part of a build or deploy process)
    Include the final configuration (without the config file) as part of your deployment artifact.
    Profit?
  • Since we’re having an amnesty session, here’s my big professional mistake.
  • Dropping a production database because a test-site was misconfigured to use it by one of the above secret management methods (you can guess)
    Let’s have some drinks after and I’ll expand on the unprofessional ones.
  • Same management principal as the config class (“know it all”), finally we add some Symfony flair!
    Use config_dev.yml and config_prod.yml to store secrets for each environment
    Largely seen in remnants of old Symfony 2.0 projects, from before the framework didn’t suggest a way of swapping values out yet (i.e. pre-parameters.ini)
    Some apps that have existed since then and just been upgraded still contain bits and pieces of this
  • No disrespect to Fabien. He’s a super smart guy and this works pretty good. But we evolve.
  • Symfony provides such a simple way forward and we all go running to do it! Let’s make everything a parameter!
    What’s wrong with this?
  • CI systems can be hacked.

    If it’s in S3 or on a server somewhere that the CI can access, can others?
    Do you audit access to this?
  • In the midst of a security incident, this may allow an unacceptable delay to roll a pivotal secret, or mean that your application “goes down” or has degraded functionality until the build/deploy completes.
    If the key is something critical (e.g. database), you can’t do a rolling deploy.
  • Lost your DB creds? You can recover them in var/cache/prod/appProdContainer.php!
    Great idea until the framework actually gained traction and then become subject to every hacker and their dog knowing how to do this.
  • Want to add a pre-prod? Or test a new feature on prod-like infra?
    New deployments now involve: updating your CI & build process, building a new artifact, updating your deployment process & tools
  • How sure are you that the issue doesn’t exist solely on production?
    Can developers drag the exact artifact down to their machine to replicate?
  • Wow this project was inconsistent with config values and quotes
  • In symfony we think of app environments, and try to keep agnostic to deployment environments (test server is still prod app environment), but for this, consider deployment envs.
  • Biggest challenge is dev environments where setting envvars is traditionally hard
  • I’ve talked a bit about security, but it’s not really. And her’es the big gotcha
  • Has the same flaws as any other parameter, plus a few new ones

×