Shut down that bastion host - there are better ways to login to your EC2 instances and Fargate containers.

1. There are better ways to
login to your EC2 instances
Shutdown that
Bastion Host
Michael Ludvig
Principal consultant @ Servian

2. Michael Ludvig
About me

3. Direct SSH to EC2
Quick and easy. And insecure.
Pros:
* Easy to set up
Cons:
* Easy to set up insecurely
* Direct internet access
* Difficult access control

4. Bastion host
For those private subnets
Pros:
* Easy to set up
Cons:
* SSH access to Bastion
* Shared SSH key
* Difficult access control

5. VPN to VPC
If you know your routes
Pros:
* More secure
Cons:
* Complicated setup
* Shared SSH key
* Difficult access control

6. Shared and re-used SSH keys
● Infamous EC2 “Key Pairs”
● Hard to audit access
● Hard to revoke and replace
● No MFA
Common problem with all the methods
Access Control
Solutions are complex to implement
● Active Directory
● Hashicorp Vault
● FreeIPA

7. IAM authentication
● Access tied to IAM Users
● Controlled by IAM policies
● Auditable in Cloud Trail
AWS native authentication
Access Control via IAM
Works with…
● SSH
● EC2 SSM Terminal Session
● EC2 Serial console
● ECS containers including Fargate

8. EC2 Session
Modern AWS-native way to login

9. SSM Session
Modern AWS-native way to login

10. SSM Session
Modern AWS-native way to login

11. EC2 Session - requirements
Modern AWS-native way to login
What do we need …
1. SSM Agent running on the EC2
○ Default in recent Amazon Linux 2 and Ubuntu 18.04+
instances
2. Network access from EC2 to SSM Service
○ Public IP, NAT gateway, Proxy, or VPC endpoint
○ No need for inbound access
3. EC2 Instance Role with AmazonSSMManagedInstanceCore
managed policy

12. SSM Session - AWS CLI example
Modern AWS-native way to login
Official AWS CLI way …
~ $ aws ssm start-session --target i-08c1950f615da4aa4
Starting session with SessionId: botocore-session-1665972406-0d66b7a6d75402362
sh-4.2$ exit
Exiting session with sessionId: botocore-session-1665972406-0d66b7a6d75402362.
… because we all remember the Instance IDs, yeah right.

13. ec2-session (a.k.a. ssm-session)
List instances:
~ $ ec2-session --profile sandpit --list
i-08c1950f615da4aa4 ec2-demo1 172.31.16.189 3.26.20.151
i-0bf539dae623eba84 ec2-demo2 172.31.104.217 # No Public IP!
Select host by name, run session as ec2-user:
~ $ ec2-session ec2-demo2 --user ec2-user
Select host by IP, run a command:
~ $ ec2-session 172.31.16.189 --command "sudo yum install -y tmux"
An easier way in

14. AWS SSM Tools
~ $ pipx install aws-ssm-tools
installed package aws-ssm-tools 1.5.0-beta5, Python 3.8.11
These apps are now globally available
- ec2-serial
- ec2-session
- ec2-ssh
- ecs-session
- ssm-tunnel
done! ✨ 🌟 ✨
Contribute on GitHub: https://github.com/mludvig/aws-ssm-tools
EC2 access for humans

15. ec2-ssh
Why?
● No need for inbound SSH access
● No need for pre-shared SSH key (aka “EC2 Key Pair”)
● Useful where ec2-session is not enough, e.g. rsync, ssh
port forwarding
● Access can be audited in CloudTrail
SSH connection over SSM session

16. [michael@MBP] ~ $ ec2-ssh --profile sandpit --send-key
-L 5432:some-rds-instance:5432 ec2-user@ec2-demo1
[michael@MBP] ~ $ ec2-ssh --profile sandpit --send-key
-L 5432:some-rds-instance:5432 ec2-user@ec2-demo1
INFO: Resolved instance name 'ec2-demo1' to 'i-08c1950f615da4aa4'
INFO: Using SSH key from SSH Agent, should be as good as any.
ec2-user@ec2-demo1 ~ $
[michael@MBP] ~ $ ec2-ssh --profile sandpit --send-key
-L 5432:some-rds-instance:5432 ec2-user@ec2-demo1
INFO: Resolved instance name 'ec2-demo1' to 'i-08c1950f615da4aa4'
INFO: Using SSH key from SSH Agent, should be as good as any.
ec2-user@ec2-demo1 ~ $ w
06:41:09 up 12 days, 21:42, 1 user, load average: 0.04, 0.09, 0.08
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
ec2-user pts/0 localhost 06:41 1.00s 0.01s 0.00s w
ec2-user@ec2-demo1 ~ $
ec2-ssh
Port forwarding example

17. [michael@MBP] ~ $ rsync -e "ec2-ssh --send-key" ec2-user@ec2-demo1:
[michael@MBP] ~ $ rsync -e "ec2-ssh --send-key" ec2-user@ec2-demo1:
INFO: Resolved instance name 'ec2-demo1' to 'i-08c1950f615da4aa4'
INFO: Using SSH key from SSH Agent, should be as good as any.
drwx------ 181 2022/06/25 22:17:57 .
-rw------- 11,455 2022/06/25 22:18:06 .bash_history
-rw-r--r-- 18 2020/01/16 13:56:01 .bash_logout
-rw-r--r-- 193 2020/01/16 13:56:01 .bash_profile
-rw-r--r-- 231 2020/01/16 13:56:01 .bashrc
[...]
ec2-ssh
RSync example

18. Login to ECS and Fargate containers
Just because you can doesn’t mean you should!

19. AWS CLI ECS Exec
Find the ECS Cluster name
nginx-Cluster-8SqXVHC
The Terminator-style CLI (i.e. not for the rest of us)
Find the Service name
nginx-Service-gL6Cnj
Find the Task name
11f37244484b486e9e0eda9b94c
Construct the AWS CLI command line:
[michael@MBP] ~ $ aws ecs execute-command --cluster nginx-Cluster-8SqXVHC
--task 11f37244484b486e9e0eda9b94c
--container nginx --command “/bin/bash” --interactive

20. ecs-session
[michael@MBP] ~ $ ecs-session --list
Cluster Group Task Cont. IP
nginx-Cluster-8SqXVHC service:nginx-Service-gL6Cnj a3875d0e...7d488 nginx 172.31.107.214
nginx-Cluster-8SqXVHC service:nginx-Service-gL6Cnj 11f37244...9b94c nginx 172.31.73.13
nginx-Cluster-8SqXVHC service:nginx-Service-gL6Cnj fb4099e3...aba46 nginx 172.31.94.23
For the non-Terminators among us (i.e. for me and you)
[michael@MBP] ~ $ ecs-session 172.31.73.13
nginx-Cluster-8SqXVHC service:nginx-Service-gL6Cnj 11f37244...9b94c nginx 172.31.73.13
Starting session with SessionId: ecs-execute-command-09c4f8dcf93299816
root@ip-172-31-73-13:/#

21. ECS / Fargate Session - requirements
Modern AWS-native way to login
What do we need …
1. Service / Task setting in CFN or Terraform:
○ EnableExecuteCommand: true
2. Task Role with AmazonSSMManagedInstanceCore managed
policy
3. Network access from the task to SSM Service
○ Public IP, NAT gateway, Proxy, or VPC endpoint
○ No need for inbound access

22. Virtual serial port
Why?
● Works even without instance connectivity
(e.g. firewall misconfiguration, boot problems)
● Boot log messages
● Last resort
Why not?
● Requires password-enabled user account
Welcome back to the 90’s

23. ec2-serial
[michael@MBP] ~ $ ec2-serial --region us-east-1 ec2-demo1
Wait a few seconds and then press Enter for a prompt...
EC2 console serial port over SSH
[michael@MBP] ~ $ ec2-serial --region us-east-1 ec2-demo1
Wait a few seconds and then press Enter for a prompt...
Amazon Linux 2
Kernel 5.10.147-133.644.amzn2.x86_64 on an x86_64
ip-172-31-16-189 login:

24. ssm-tunnel
[michael@MBP] ~ $ ssm-tunnel ec2-demo1 --route 172.31.0.0/16
[sudo] password for mludvig: ****
INFO: Local IP: 100.64.156.100 / Remote IP: 100.64.156.101
00:00:34 | In: 472.0 B @ 5.2 B/s | Out: 2.7kB @ 0.0 B/s
Your ad-hoc VPC backdoor
[michael@MBP] ~ $ ping 172.31.104.217 # ec2-demo2
PING 172.31.104.217 (172.31.104.217) 56(84) bytes of data.
64 bytes from 172.31.104.217: icmp_seq=1 ttl=254 time=90.8 ms
64 bytes from 172.31.104.217: icmp_seq=2 ttl=254 time=89.0 ms
64 bytes from 172.31.104.217: icmp_seq=3 ttl=254 time=91.4 ms
^C
--- 172.31.104.217 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 89.043/90.447/91.413/1.073 ms

25. Get in touch!
https://www.linkedin.com/in/michaelludvig
AWS SSM Tools:
https://github.com/mludvig/aws-ssm-tools
Blog posts about (not only) AWS SSM
Tools:
https://michael-ludvig.medium.com