This document discusses Debian cloud images for Amazon EC2. It provides an overview of Debian, AWS EC2, different types of EC2 machine images including EBS-backed and S3-backed AMIs. It describes how Debian generates and distributes its official Debian AMIs across different AWS regions and architectures while managing the image lifecycle and security updates. It also briefly mentions using Cloudfront CDN for Debian archive access from EC2 instances.

1. Debian-Cloud: EC2 AMIs
James Bromberger
<jeb@debian.org>

2. Agenda
• What is Debian
• What is AWS EC2
• A meander through block storage for EC2
instances
• Types of images
• Generating & distributing Debian’s AMIs
• Debuab Image lifecycle and security
• If there is time: Debian via Cloudfront CDN

3. WHAT IS DEBIAN

4. What is Debian
• Computer Operating System
– 14 CPU/kernel architectures
– 37,500 packages of software
– Translated into a bunch of languages

5. What is Debian
• Primarily of free and open-source software
– GNU General Public License and many other
licenses

6. What is Debian
• Started 1993
– 21 years old now
• Democratic, volunteer organisation - ~1,000 people
(please join!)
– Zero payed employees

7. WHAT IS AWS EC2

8. What is AWS and EC2
• AWS = Amazon Web Services
• EC2 = Elastic Compute Cloud
– Virtual servers running Linux, Windows, BSD
• Started 2006
• Now with 11 Regions and 52 Edge Locations
• Compute, storage, platform, infrastructure – as-a-service
– typically billed by the hour or by the month
Amazon EC2

9. What is EC2
• Compute requires:
– CPU, Memory (RAM)
– Block Storage (disk)
– Network
– Automation & bootstrapping
– Self-service
instance
Amazon EBS
Amazon VPC

10. What is EC2
• Amount of CPU & Memory is combined into
“instance type”:
– Small
– Medium
– Large
– ...
instance
instance
instance

11. What is EC2
• Several instance types are grouped into an
“instance family”:
– General Purpose (balanced memory:cpu)
– Memory Optimised (more memory:cpu)
– CPU Optimised (more cpu:memory)
– Storage Optimised (more ‘ephemerial’ storage)
– GPU (CUDA, OpenCL)
– Cluster Nodes (10 GB/sec networking and more)

12. What is EC2
• EC2 instance run on real servers!
instance instance instance instance
Total number of
(hyperthread)
CPU cores, each
dedicated* to an
instance
Disk inside the
physical server is
deemed
‘ephemeral’. Not
raid, but is local to
CPU and Memory.
Different amounts
of storage
depending on
instance type
RAM is dedicated
to each instance
Each instance can
send a certain
number of packets
per second

13. A MEANDER THROUGH STORAGE

14. Ephemeral (instance) Storage
instance instance instance instance

15. Persistent (EBS) Storage
instance instance instance instance
Amazon EBS

16. Persistent (EBS) Storage
instance instance instance instance
Amazon EBS

17. Persistent (EBS) Storage
Amazon EBS
Mechanical disk
General Purpose SSD (GP2)
Provisioned IOPS (SSD)
Amazon S3

18. Persistent (EBS) Storage
Amazon EBS
Mechanical disk
General Purpose SSD (GP2)
Provisioned IOPS (SSD)
Amazon S3
AFR of a typical standard HDD
Designed for 99.999% availability
(5.26 min/yr)
Single instance attach only
(currently)
1GB..1TB (currently)
Your choice of file-system
Optional transparent encryption
by AWS
Network attached to your
instance back in the EC2
environment
99.999999999% durability
Replicated multiple times
within the same Region
Check-summed and re-check-
summed periodically
Designed for 99.99%
availability (SLA at 99.9%)
Can be shared with other
customers (specific, or all)
unless AWS-encrypted
Can be used to create a
new EBS volume
EBS snapshots cannot be
seen in your S3 buckets

19. Persistent (EBS) Storage
Amazon EBS
Mechanical disk
General Purpose SSD (GP2)
Provisioned IOPS (SSD)
Amazon S3
AFR of a typical standard HDD
Designed for 99.999% availability
(5.26 min/yr)
Single instance attach only
(currently)
1GB..1TB (currently)
Your choice of file-system
Optional transparent encryption
by AWS
Network attached to your
instance back in the EC2
environment
99.999999999% durability
Replicated multiple times
within the same Region
Check-summed and re-check-
summed periodically
Designed for 99.99%
availability (SLA at 99.9%)
Can be shared with other
customers (specific, or all)
unless AWS-encrypted
Can be used to create a
new EBS volume
EBS snapshots cannot be
seen in your S3 buckets

20. Persistent (EBS) Storage
instance instance instance instance
Amazon EBS

21. Instance stop w/EBS
instance instance instance
Amazon EBS

22. Instance restart w/EBS
instance instance instance
Amazon EBS
instance instance instance instance
EBS volume(s) reattached, ephemeral volume(s) blank

23. TYPES OF MACHINE IMAGES

24. Amazon Machine Images
• AMI is “golden master”
• Start as many instances as you like*
AMI
instance
instance
instance
instance instance instance

25. Ephemeral and EBS
• Why is the Ephemeral and EBS storage options
important in AMIs?
Your root volume
/ -> persistent (EBS)
/ -> transitory (Ephemeral)

26. Ephemeral and EBS
• Why is the Ephemeral and EBS storage options
important in AMIs?
Your root volume
1,000 systems for 24 hours,
8 GB EBS each in SYD: ~$30.85

27. Ephemeral and EBS
• Why is the Ephemeral and EBS storage options
important in AMIs?
Your root volume
1,000 systems for 24 hours,
Ephemeral in SYD: $0

28. Ephemeral and EBS
• Why is the Ephemeral and EBS storage options
important in AMIs?
Amazon S3 S3 backed AMI
snapshot
EBS backed AMI

29. CPU Architectures
• EC2 currently supports 2 architectures:
EBS backed AMI S3 backed AMI EBS backed AMI S3 backed AMI

30. Virtualisation Types
• EC2 uses (highly customised) Xen, and
supports two virtualisation types:
Para-
Virtualization
(threads)
Hardware
Virtualization
(emulation)
EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI

31. Each Region is independent
Para-
Virtualization
(threads)
Hardware
Virtualization
(emulation)
EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI
EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI
EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI EBS backed AMI S3 backed AMIEBS backed AMI S3 backed AMI
AP... US West 1 US East 1

32. Now multiply that by:
• Wheezy
• Jessie
• Sarge
• ...

33. • 2 architectures
• 2 virtualisation types
• 2 root volume types
• 11 Regions
• 3 Debian releases
= 198 images
(Plus images currently being end-of-lifed,
experimented with, and used for other purposes)

34. Current Debian AMIs: Squeeze (6)
Architecture EBS Backed S3 Backed
32 bit PVM Yes
64 bit PVM Yes
32 bit HVM
64 bit HVM

35. Current Debian AMIs: Wheezy (7)
Architecture EBS Backed S3 Backed
32 bit PVM Yes
64 bit PVM Yes Yes
32 bit HVM
64 bit HVM Yes (experimental)

36. Future Debian AMIs: Jessie (8)
Architecture EBS Backed S3 Backed
32 bit PVM
64 bit PVM Yes
32 bit HVM
64 bit HVM Yes Yes*

37. Two ways of creating AMIs
Start from scratch
• Uses a fresh, blank volume,
install as a debootstrap
Update existing
• Start existing instance,
customise, create new
image

38. EBS Backed AMI overview
instance
volume
/
volume
/target
snapshot
EC2 API
Endpoint
AMI

39. Let’s create a Jessie image
• Fire up an existing instance (easiest is to use
an existing Debian AMI)
• Install git, debootstrap, python-boto, python-jsonschema,
and some other python bits
– Configure your AWS IAM credentials for boto
• Grab bootstrap-vz from Github

40. DEMO

41. Distributing images globally
Each region has separate copies of AMIs

42. Distributing images
Three “groups” of Regions:
• GovCloud
• Beijing
• Everywhere else*

43. Debian AWS Accounts
Region AWS Account ID
Beijing 673060587306*
Gov Cloud 256493402735**
Standard Regions 379101102735

45. Community Shared AMIs
• Un-vetted by AWS
– Trojan horses
– Left over SSH keys in other accounts
– Cron jobs that go bump in the night
• Anyone can share any AMI under their control
(provided they have access within their AWS account to do so – IAM Policy)
– Caveat emptor

48. Pushing images to Marketplace
Vendor
AWS
Account ID
Vendor
Display
Name
Product ID Version ID ASIN SKU Software
by
Title Version
Title
Release
Notes
Short
Description
Description Highlight1

50. IMAGE LIFECYCLE AND SECURITY

51. AMI Lifecycle
Our aim is to keep the final point release AMI
available for each Debian major release,
starting from Squeeze:
• 6.0.10
• 7.7

52. AMI Lifecycle
Wheezy 7.4
Wheezy 7.5
Try to keep a 2 – 5
week overlap for point
releases, then un-share
Wheezy 7.6
for a period,
Wheezy
7.6.aws.
1
Wheezy
7.6.aws.2
Wheezy 7.7
then delete
Time
Occasionally security
releases that are urgent
in BASE images (AMIs)
force additional version
numbers out of step with
Debian. This was
shellshock,

53. Security in base images
• EC2 instances may be deployed such that they
don’t have direct access to fetch updates
• Administrators may chose not to install
updates unattended

54. Debian AMIs in US East 1

55. Workflow overview
1. Generate AMIs in US East 1
2. Tag AMIs and Snapshot
3. Test image in US East 1
4. Copy to all Standard Regions (python script)
5. Mark AMI and Snapshot as Public (python script)
6. Generate in Beijing and Gov Cloud, tag, mark public
7. Generate signed message to the Debian-cloud mailing list, update wiki
8. Wait a few days (for bugs to surface), then push to AWS Marketplace
9. Announce deprecation of previous versions (typically 3 – 5 weeks notice)
in signed email to Debian-cloud ML
10. After elapsed period, remove public sharing from AMI and Snapshots
(python script)
11. A day or so later, deregister the AMI and delete the snapshot (python
script)

56. What’s new in Jessie EC2 images
• Single Root IO Virtualisation (Enhanced
Networking)
• Multiple Network Interfaces (ENI)
• Multiple sub-interfaces
• AWS CLI and python-boto installed in base
image
• Cloud-init (since Wheezy 7.4)

57. Cloud-init
• Insert this as “User
Data”
• Can be embedded into
CloudFormation
templates
#cloud-config
package_update: true
package_upgrade: true
package_reboot_if_required: true
packages:
- pwgen
- less
locale: fr_FR.UTF-8
ssh_authorized_keys:
- ssh-rsa AAAAB3Nz....89dGp5 me@mykey1
- ssh-rsa AAAAB3Nz....89dGp5 me@mykey2
final_message: "The system is finally up,
after $UPTIME seconds"

58. DEBIAN ON CLOUDFRONT CDN

59. Debian Archive via CDN
• Default apt sources.list
for EC2 images uses
cloudfront.debian.net
• Primarily for EC2
instances, but is active
in all 52 Cloudfront
locations world-wide
CloudFront

60. Cloudfront.debian.net
• Each edge location is
independent of all
others
edge location
edge location
edge location
traditional server

61. Cloudfront.debian.net
• However, Debian HTTP
servers don’t put any
cache advisory headers
on how long objects
(files) may be cached
for; some of these are
quite volatile, and some
are very stable
edge location
edge location
edge location
traditional server

62. Cloudfront.debian.net
• Luickly, Cloudfront
supports “Cache
behaviours”, mapping
different URL paths to
alternate origin servers
edge location
edge location
edge location
traditional server

63. Cloudfront.debian.net
• Default: => S3 bucket
• /debian/ => ftp.us.debian.org
• /debian/dists => my proxy server edge location
edge location
edge location
bucket
http://ftp.us.debian.org
Elastic Load instances
Balancing

68. Jessie on Ice
(ie, Jessie is frozen)

69. Debian 9:
Stretch
Debian 10: Buster
1.1 (‘96)
1.2
1.3
2
2.1
2.2
3 (2002)
3.1
4
5
6
7
8 (2015)

70. James Bromberger
E: jeb@debian.org
T: @JamesBromberger
L: https://www.linkedin.com/in/jamesbromberger
Perth, Australia
GPG: 4096R/9D85C53C 2011-11-29
Key fingerprint = 8591 20FE 0D9F A6A5 B054 C775 AEC8 2874 9D85 C53C
https://github.com/JamesBromberger/bootstrap-vz
https://wiki.debian.org/Cloud/AmazonEC2Image
https://aws.amazon.com/marketplace/pp/B00AA27RK4
https://lists.debian.org/debian-cloud/