by Andrew Kiggins, Solutions Architect, AWS

1. © 2016 AWS and affiliates, all rights reserved
Security Architecture Loft
Architecting For Resiliency
Andrew Kiggins Security Solution Architect
Aren Sandersen, CTO, FoxPass

2. © 2016 AWS and affiliates, all rights reserved
Goals
• Learn how to control access
• Learn how AWS services can help implement fault tolerance
• Learn how AWS services can reduce security incident blast radius
• Understand the security services and features associated with network
configuration
• Understand how to secure servers and control user access

3. © 2016 AWS and affiliates, all rights reserved
Conditions affecting service availability
• Large Scale Events (LSEs)
• Disaster recovery
• DDoS Attacks
• Deployment failures

4. © 2016 AWS and affiliates, all rights reserved
VPCs
• Public + Private
subnets
• IGW
• NAT GW
virtual private cloud
VPC NAT
gateway
Availability ZoneAvailability ZoneAvailability Zone
VPC subnetVPC subnetVPC subnet
VPC subnetVPC subnetVPC subnet
Internet
gateway

5. © 2016 AWS and affiliates, all rights reserved
VPCs
• Security Groups
– Stateful
– Ingress/Egress
– Instance specific
– Default deny
• Network ACLs
– Stateless
– Inbound/
Outbound
– Subnet specific
– Default allow virtual private cloud
Availability Zone
VPC subnetVPC subnet
Internet
gateway
instanceinstance
instanceinstance
instance
security group
security groupsecurity group

6. © 2016 AWS and affiliates, all rights reserved
VPC
security group
HTTP GET
TCP(6) Port(80)
NTP Buffer Overrun
UDP(17) Port(123)

7. © 2016 AWS and affiliates, all rights reserved
Network ACL
security group
HTTP GET
TCP(6) Port(80)
HTTP GET
TCP(6) Port(80)
srcIP=216.246.16.228

8. © 2016 AWS and affiliates, all rights reserved
Obfuscate
VPC
Amazon
Route 53
CloudFront
users
security group (BuildABeer-SG-1)
Public subnet
servers
Private subnet
ELB

9. © 2016 AWS and affiliates, all rights reserved
Layers Of Access Control
users
Application
servers
Private subnet
security
group
Public subnet
ELB IPS/IDS
Private subnet
ELB
security group
WAF
Private subnet
ELB
security group security group
ELB

10. © 2016 AWS and affiliates, all rights reserved
Availability Zones
• 2 zones good, 3 zones better
• Remove AZ dependencies
– AZs can and should stand alone

11. © 2016 AWS and affiliates, all rights reserved
Regions (14+)
• Regional Isolation
– Keys don’t port
• Insulate from regional dependency
– Reduce centralized resources
• Reduce Blast Radius
– Don’t interconnect regions
– No VPNs
– Use protected API endpoint for communication between regions
• Disaster recovery

12. © 2016 AWS and affiliates, all rights reserved
Elastic Load Balancer + Application Load Balancers
• Cross AZ
– Fault tolerance
• Auto Scaling
– Load/attack resilient
• Service isolation
Elastic Load Balancing

13. © 2016 AWS and affiliates, all rights reserved
Separation of services
Amazon
Route 53
CloudFront
security group
Public subnet
servers
Private subnet
ELB
www.foo.commail.foo.com
security group
Public subnet
Mail servers
Private subnet
ELB
security group
Public subnet
Web servers
Private subnet
ELB
mail.foo.com
www.foo.com

14. © 2016 AWS and affiliates, all rights reserved
EC2 Auto-scaling
• Create Auto Scaling Groups to span subnets & AZs
• Scale up quickly (in jumps)
• Scale down slowly (in steps)

15. © 2016 AWS and affiliates, all rights reserved
Introduction
• About me
• Now: Founder, Foxpass, DevOps security company
• Then: Led operations, IT, and security teams for startups over a decade
• DevOps/Security consulting

16. © 2016 AWS and affiliates, all rights reserved
Goals
• Understand where AWS leaves off and your own security practices must begin,
particularly for Linux servers
• Learn common threats and how to decrease exposure
• Get practical advice on how to configure production access and set IT policies

17. © 2016 AWS and affiliates, all rights reserved
Amazon is easy!
• Amazon’s AWS portal makes it extremely simple to bring up servers and get
started
• Creating firewalls is easy, too
– Security groups
– Network ACLs in VPCs
– WAFs
• IAM is extremely granular for portal/api access

18. © 2016 AWS and affiliates, all rights reserved
But more is needed, especially at the machine level
• AWS defaults to public-private SSH key on non-root user (ubuntu, ec2-user)
which encourages sharing of the private key
• Sharing a private key is not a best practice
– If an employee leaves, now you have to rotate that key on every server
– This is annoying
– This is dangerous and can lead to lockouts
• AWS defaults to a single user on each Linux machine
– Sharing user accounts is (usually) not a best practice
• AWS does not give you a bastion host or a VPN
– It’s up to you

19. © 2016 AWS and affiliates, all rights reserved
How do I improve this?
• Bare minimum:
– Update the authorized_keys file for your common user (ubuntu, ec2-user) with the
public ssh key(s) of each employee who can log in
– Every device should have a separate keypair:
ubuntu@staging:~$ cat ~/.ssh/authorized_keys
ssh-rsa AAAAB3... 7EFqEu aren@aren-macbook
ssh-rsa AAAAB1... 8EcZXy aren@aren-ipad
ssh-rsa AAAAB8... HG2AZ6 dave@laptop
. . .
. . .

20. © 2016 AWS and affiliates, all rights reserved
How do I improve this further?
• Get rid of shared user accounts
– See who exactly is on a machine
– See what they are doing
– Assign different permissions to different users
• Who can log in
• What can they see/do when they log in
– Auditable

21. © 2016 AWS and affiliates, all rights reserved
Ok, how?
• Add by hand with useradd command
– Works fine for one machine, does not scale (user id numbers don’t line up)
• Use configuration management like Chef, Puppet, OpsWorks
• Use centralized directory server (LDAP)
– Run your own
– Use a SaaS provider

22. © 2016 AWS and affiliates, all rights reserved
SSH access to production
• Don’t keep port 22 open to the world to every host
• Instead:
– Use a bastion host
• Users SSH to the bastion host (jump box) and from there can access another host via the private IP
(10.x.x.x)
– Use a VPN server
• Makes employee laptop a part of the production network
• OpenVPN (opensource and Access Server)
– Requires special client
• Ipsec-based can use built-in client
– Foxpass gives one of these away in the marketplace
• No matter which you choose, requiring 2-factor for access to your production
infrastructure is critical
– Google Authenticator
– Duo

23. © 2016 AWS and affiliates, all rights reserved
IT security
• Easiest way to break into a company is by tricking the employees
– Two-factor authentication reduces attack surface and should be required for email
accounts
• Laptop security
– SSH keys should be created with passwords (use ssh-add so you don’t have to re-
type the password)
– Enforce laptop disk encryption from day 1
– Turn on locking screen savers

24. © 2016 AWS and affiliates, all rights reserved
IT security, continued
• Network security
– If your office IP is whitelisted into production anywhere, you need to secure your
office wireless
• Consider not whitelisting it: treat your office like any coffee shop
• Whitelists to back-office apps are the worst, many app security holes
• Make p/w hard to remember and do not write it on whiteboards (for example)
• Rotate the password frequently, especially after an employee leaves
– Or set up 802.1x
• 802.1x forces the OS to ask a user for his/her name and password before granting access to
the WiFi network
• Needs a RADIUS server. Run your own or use a SaaS provider

25. © 2016 AWS and affiliates, all rights reserved
Have an employee departure checklist
• Servers
• VPN
• Google Apps account
• AWS account
• Domain Registrar
• DNS

26. © 2016 AWS and affiliates, all rights reserved
Thank you!
aren@foxpass.com