This presentation details how we, at OpenSesame, have built a system to deliver a platform and core services to our engineering teams. It details the structure of what each engineering team gets and how they access it.

1. Account Bootstrap
OpenSesame DevOps

2. Motivation
● More teams 🡒 more services 🡒 more risk
● Multiple vendor accounts
○ Minimizes blast radius
○ Enhances Principle of Least Privilege
○ Provides strong service boundaries
○ Allows teams to deploy more independently
● A layered vendor account architecture
○ Allows centralized teams (DevOps, SecOps) to provide foundational resources
■ Terraform state management
■ Off-site backups
■ Log management
■ Access controls
■ Networking stack

3. Outline
● Vendor Accounts
● Account Roles
● Account Login
○ First-Time Setup
○ Day-to-Day Usage

4. Vendor Accounts

5. Terminology
● Each team supports one or more “product domains”
(“domain” for short)
○ Examples: Data Science, Catalog
● Each product domain receives:
○ A set of AWS, New Relic accounts; one per “account
class”
○ User credentials
○ Starter-pack of pre-configured resources

6. Accounts
● Dev
○ For sandbox play, and development on-platform
● Stage
○ Matches production resources closely
○ DB contains fake or sanitized prod data
○ For load/scale testing, cross-team or external integration testing
● Prod
○ All external customer-facing resources

7. Starter Pack
● Account KMS key
○ Auto-rotating
○ Use for any encryption needs (DBs, Redis, secrets, SQS/SNS)
● Terraform state
○ S3 state bucket, DynamoDB lock table
○ Encrypted with account KMS key
● Faceless users
○ Users who are not a real, live person
○ Used for services like CI/CD, security scanning, etc.
○ Example: DataScienceGitHubActionsFacelessUser
○ Credentials available in Secrets Manager (Someday: auto pushed to CI provider)

8. Starter Pack
● Networking stack
○ VPC with 2 subnets (multi-AZ), NAT Gateways, and Internet Gateway
● Logging & Metrics
○ AWS account pre-wired to forward metrics to New Relic
○ S3 bucket for Load Balancer access logs (forwards to New Relic)
○ Lambda for forwarding CloudWatch logs

9. Account Roles

10. Users
● (New) Each team member gets a single AWS user
○ Use for all AWS access for that person
● Each user has 1 set of Access keys
○ Used for CLI access
○ No need to store multiple keys in
~/.aws/credentials

11. Roles
● Each AWS user is granted permissions via IAM Roles
○ Roles are like hats - assume the right one for the job
● Two roles per account
○ Developer - day to day use
○ Operator - account admin-level privileges
● Teams (ie: team leads) decide which team members
are Developers vs. Operators
○ For small teams - everyone can be an Operator

12. By default, Developers have:
● Full access to all domain Dev
accounts (open-source model)
● Read access to their domain’s
Stage and Prod accounts
● No IAM access to other
domain Stage and Prod
accounts
All permissions can be extended by
the team.
Developers can only assume
Developer roles.

13. By default, Operators have:
● Full access to all domain Dev
accounts (open-source model)
● Full access to their domain’s
Stage and Prod accounts
● No IAM access to other
domain Stage and Prod
accounts
Operators can assume Developer or
Operator roles, depending on their
task.

14. Faceless Users
● Each gets a role - team sets permissions
● Created in global AWS account - NOT in domain accounts
○ Existing users will be migrated over time
● Example: DataScienceProdGitHubActionsFacelessRole

15. Admins
● Users requiring broad, all-domain access to perform their work
○ Examples: Directors, CTO, DevOps, SecOps
● Can assume any Developer or Operator role
○ Includes non-domain accounts like Security Portal, Audit

16. Account Logins
Web Console Command Line

17. Account Login -
Web Console

18. 1. Password Reset
● Admin creates your user
● Log on to AWS Console
○ Account ID: global-aws-account-id
○ User: your.name as provided
○ Password: as provided
● Change your password
○ 1 week grace period to reset from account creation
○ Use a password manager please :)

19. AWS Role Switcher
If it displays:
● your.name @ opensesame-…
○ No role assumed
○ Can change password, MFA
● Role Name
○ Role assumed
○ Use Back to your.name to
un-assume role

20. 2. Set up MFA
● From the login screen:
○ Click on Role Switcher menu
○ Select “My Security Credentials”
○ Click “Manage MFA Device” button
○ Select the Virtual MFA Device option
○ Follow the instructions

21. 3. Set up AWS Extend Role Switcher Plugin
● Install plugin:
○ For Chrome
○ For Firefox
● Open plugin 🡒 “Configuration”
○ Copy-paste from here
○ Select Local option for Configuration storage
○ Customize as desired (change colors, icons, ordering,
remove unused roles, etc)

22. 4. Log out, Log in
● Log out and in to kick MFA
○ Click on your.name 🡒
○ Select “Sign Out”
○ Log back in
● Click on Role Switcher Plugin (Blue Key) 🡒
Choose a role!

23. Day-to-Day Login
● Log on to https://console.aws.amazon.com
○ Account ID: global-aws-account-alias
○ User: your.name as provided
○ Password: as provided
● Assume a role using the Role Switcher browser
plugin (Blue Key)

24. Account Login -
Command Line

25. 1. Install aws-mfa locally
● (New) MFA required for command-line access
● aws-mfa automates fetching and caching AWS tokens
○ Token valid for 12 hours - refresh once per work day
● Install Option 1: Via pip3
○ pip3 install aws-mfa
● Install Option 2:
○ Git clone https://github.com/broamski/aws-mfa.git
○ python3 setup.py install

26. 2. Generate AWS CLI Credentials
● From any AWS console page:
○ Click on Role Switcher menu
○ Select “My Security Credentials”
■ Click “Back to your.name” if not visible
○ Click the “Create access key” button, and leave the page
open
[portal-long-term]
aws_access_key_id = <Your AWS access key ID>
aws_secret_access_key = <Your AWS secret key>
aws_mfa_device = <Your MFA device ARN>

27. 3. Edit credentials file
● With your favorite text editor:
○ Open ~/.aws/credentials
○ Copy-paste from here
○ Replace values in top section with
your AWS access key ID and secret,
and your user.name
[portal-long-term]
aws_access_key_id = <YOUR AWS ACCESS KEY ID>
aws_secret_access_key = <YOUR AWS SECRET ACCESS KEY>
aws_mfa_device = arn:aws:iam::000000000000:mfa/<YOUR AWS user.name>
[devops-prod]
source_profile = portal
role_arn = arn:aws:iam::000000000000:role/DevOpsProdDeveloperRole
[devops-prod-ops]
source_profile = portal
role_arn = arn:aws:iam::000000000000:role/DevOpsProdOperatorRole
[data-science-dev]
source_profile = portal

28. Day-To-Day Login
● Open a terminal, run aws-mfa --profile portal
○ Enter 2FA code if prompted
● Run aws commands, terraform, etc.
○ NOTE: Expired tokens can cause weird errors. If you get a weird error, be sure to check aws-mfa.

29. Review

30. Review
● AWS Accounts
○ Terminology
○ Dev, Stage, Prod
○ Starter Pack Resources
● Account Roles
○ Developer
○ Operator
● Account Login
○ Web Console
○ AWS CLI

31. Thank You
Questions?