Arquitetura Hibrida - Integrando seu Data Center com a Nuvem da AWS

1. Michel Pereira
Hybrid Architecture: Integrating your Data Center with AWS
Enterprise Solutions Architect

2. Vários Tutoriais , treinamentos e mentoria em
português
Inscreva-se agora !!
http://awshub.com.br

3. Datacenters limitations
How much time to deliver a project?

4. Capacity
Datacenter

5. Agility
Datacenter
Requirement
Server
Load Balancer
Firewall
Storage
Approvals
Hardware or VM
aquisition
Capacity allocation
Ticket queuing
Provisioning
Configuration
Request
Availability!!!

6. Capacity is a premium resource...

7. ...and take time to have!

8. Cost
Datacenter
Project Two
Potential impact: HIGH
Potential cost: HIGH
Project One
Potential impact: LOW
Potential cost: HIGH
Project Three
Potential impact: LOW
Potential cost: LOW
DENIED
DENIED
APPROVED

9. Cost of infrastructure can inhibit innovation

10. And now comes an era of an Elastic Data Center
Flexible, on-demand facilities

11. Cost
AWS
Project Two
Potential impact: HIGH
Potential cost: HIGH
Project One
Potential impact: LOW
Potential cost: HIGH
Project Three
Potential impact: LOW
Potential cost: LOW
APPROVED
APPROVED
APPROVED
COMPLETED
COMPLETED
COMPLETED

12. And now comes an era of an Elastic Data Center
Flexible, on-demand facilities

13. That you can integrate with your on-premises
No pain, only gain

14. Regions
Where?

15. US-WEST (Oregon)
EU-WEST (Ireland)
ASIA PAC (Tokyo)
ASIA PAC
(Singapore)
US-WEST (N. California)
SOUTH AMERICA (Sao Paulo)
US-EAST (Virginia)
AWS GovCloud (US)
ASIA PAC (Sydney)
Regions

16. US-WEST (Oregon))
EU-WEST (Ireland)
ASIA PAC (Tokyo)
ASIA PAC
(Singapore)
US-WEST (N. California)
SOUTH AMERICA (Sao Paulo)
US-EAST (Virginia)
AWS GovCloud (US)
ASIA PAC (Sydney)
Availability Zones

17. VPC

18. Amazon Virtual Private Cloud (Amazon VPC) enables you to
launch Amazon Web Services (AWS) resources into a virtual
network that you’ve defined.
1What is
Amazon
VPC?

19. Bridge your VPC and your
onsite IT infrastructure with an
encrypted VPN connection
Store data in S3 and set
permissions to allow access
only from within your VPC
Assign multiple IP address and
attach multiple ENIs and EIPs
to EC2 instances
Control Inbound and outbound
access to and from individual
subnets
Specify your own private IP
address range from any
ranges you choose
Divide your private IP address
range into one or more public
or private subnets.
What is VPC?

20. Bridge your VPC and your
onsite IT infrastructure with an
encrypted VPN connection
Store data in S3 and set
permissions to allow access
only from within your VPC
Assign multiple IP address and
attach multiple ENIs and EIPs
to EC2 instances
Control inbound and outbound
access to and from individual
subnets
Specify your own private IP
address range from any
ranges you choose
Divide your private IP address
range into one or more public
or private subnets.
Only cost is
optional hardware
VPN
What is VPC?

21. Overview of VPC
Public Subnet
10.0.0.0/24
Internet Customer Network
172.16.0.0/8
Web Server
10.0.0.5
198.51.100.1 (EIP)
Web Server
10.0.0.6
198.51.100.2 (EIP)
Web Server
10.0.0.7
198.51.100.3 (EIP)
NAT
10.0.0.8
198.51.100.4 (EIP)
Private Subnet
10.0.1.0/24
DB Server
10.0.1.5
DB Server
10.0.1.6
DB Server
10.0.1.7
Amazon EC2
API endpoint
Amazon S3
API endpoint
R
VPC
10.0.0.0/16
Custom Route Table
Destination Target
10.0.0.0/16 local
0.0.0.0/0 Internet Gateway
Main Route Table
Destination Target
10.0.0.0/16 local
172.16.0.0/8 Virtual Private Gateway
0.0.0.0/0 NAT Instance
Internet
Gateway
VPN
Gateway

22. With Amazon VPC, you can define a virtual network topology that
closely resembles a traditional network that you might operate in
your own data center.
2Subnets,
gateways,
and routes

23. • Select a region for your own network in the cloud.
Availability ZoneAvailability Zone Availability ZoneAvailability Zone
Insert network here.

24. • Select a region for your own network in the cloud.
• Create your own subnet (s).
Availability ZoneAvailability Zone Availability ZoneAvailability Zone
VPC Subnet 1 VPC Subnet 2 VPC Subnet n

25. • Select a region for your own network in the cloud.
• Create your own subnet (s).
• Configure custom routing rule(s).
Availability ZoneAvailability Zone Availability ZoneAvailability Zone
VPC Subnet 1 VPC Subnet n

26. • Availability zone boundary
• Routing table boundary
• Network access control boundary
Availability ZoneAvailability Zone Availability ZoneAvailability Zone
VPC Subnet 1
VPC Subnet 4
VPC Subnet 3
VPC Subnet 2

27. • Routing traffic
• Virtual Private Gateway
• Internet Gateway
Availability Zone
VPC Subnet 1
VPC Subnet 2
Corporate Data center

28. • Virtual Private Gateway (VGW)
• IPSEC VPN
Availability Zone
VPC Subnet 1
VPC Subnet 2
VPN GatewayRoute Table Route Table
Corporate Data center

29. • Virtual Private Gateway (VGW)
• IPSEC VPN
• AWS Direct Connection
Availability Zone
VPC Subnet 1
VPC Subnet 2
AWS Direct
Connect
VPN GatewayRoute Table
Route Table
Corporate Data center

30. • Internet Gateway (IGW)
• IPSEC VPN
Availability Zone
VPC Subnet 1
VPC Subnet 2
VPN GatewayRoute Table Route Table
Route Table Internet Gateway
Corporate Data center

31. • Virtual Private Gateway (VGW)
• Internet Gateway (IGW)
Availability Zone
VPC Subnet 1
VPC Subnet 2
VPN GatewayRoute Table Route Table
Route Table Internet Gateway
Corporate Data center

32. Good, got my VPC. And?
Welcome to the cloud, I’ll be your guide

33. Amazon Elastic Compute Cloud (EC2)
• Resizable compute capacity
• Complete control of your computing resources
• Reduces the time required to obtain and boot new server instances to minutes
• Scale capacity as your computing requirements change
• Pay only for capacity that you actually use

34. EC2 Instances
Family Use
Standard Have memory-to-CPUratios suitable for most general-purposeapplications.
Micro Provide a small amount of consistent CPU resourcesand enable you to burst CPU capacity when additional
cycles are available. They're well-suited for lower throughput applications and websites that consume
significant compute cycles periodically.
High Storage Provide very high storage density and high sequential read and write performanceper instance. They are well-
suited for data warehousing, Hadoop/MapReduce,and parallel file systems.
High Memory Have proportionally more memory resources.They're well suited for high-throughputapplications, such as
database and memory caching applications.
High Memory Cluster Have large amounts of memory coupled with high CPU and network performance.These instances are well
suited for in-memory analytics, graph analysis, and scientific computing applications.
High I/O Provide tens of thousandsof low-latency, random I/O operationsper second (IOPS) to an application. They're
well-suited for NoSQL databases, clustered databases, and OLTP (online transaction processing) systems.
High CPU Have proportionally more CPU resourcesthan memory (RAM). They'rewell-suited for compute-intensive
applications.
Cluster Compute Have a very large amount of CPU coupled with increased networking performance.They'rewell-suited for High
PerformanceCompute (HPC) applications and other demanding network-bound applications.
Cluster GPU Provide general-purposegraphicsprocessing units (GPUs), with proportionally high CPU and increased network
performancefor applications that benefit from highly parallelized processing. They'rewell-suited forHPC
applications as well as renderingand media processing applications.

35. Standard
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
M1 Extra Large 15 GiB 8 4 (with 2 ECUs
each)
1680 GB (4 x
420 GiB)
64-bit High Yes m1.xlarge
M1 Large 7.5 GiB 4 2 (with 2 ECUs
each)
840 GiB (2 x
420 GiB)
64-bit Moderate Yes m1.large
M1 Medium 3.75 GiB 2 1 400 GiB (1 x
400 GiB)
32-bit and 64-bit Moderate Yes m1.medium
M1 Small 1.7 GiB 1 1 150 GiB (1 x
150 GiB)
32-bit and 64-bit Moderate Yes m1.small
M3 Double Extra
Large
30 GiB 26 8 (with 3.25 ECUs
each)
EBS storage only 64-bit High Yes m3.2xlarge
M3 Extra Large 15 GiB 13 4 (with 3.25 ECUs
each)
EBS storage only 64-bit Moderate Yes m3.xlarge

36. Micro and High CPU
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
Micro 615 MiB Up to 2 (for
short
periodic
bursts)
1 None (use
Amazon EBS
volumes for
storage)
32-bit and
64-bit
Low Yes t1.micro
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
High-CPU
Extra Large
7 GiB 20 8 (with 2.5
ECUs each)
1680 GiB (4
x 420 GiB)
64-bit High Yes c1.xlarge
High-CPU
Medium
1.7 GiB 5 2 (with 2.5
ECUs each)
340 GiB (1 x
340 GiB)
32-bit and
64-bit
Moderate Yes c1.medium

37. High Storage and High I/O
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
High
Storage
Eight Extra
Large
117 GiB 35 16 (8 cores
+ 8
hyperthread
s)
48 TiB (24 x
2 TiB hard
disk drives)
64-bit Very high
(10 Gbps
Ethernet)
No hs1.8xlarge
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
High I/O
Quadruple
Extra
Large***
60.5 GiB 35 8 (with 4.37
ECUs each)
2 TiB (2 x 1
TiB SSD)
64-bit Very high
(10 Gbps
Ethernet)
No hi1.4xlarge

38. High Memory
Name Memory Compute
Units
Virtual Cores Instance
Store
Volumes
Architecture I/O
Performance
Available for
Spot
Instance
API Name
High-
Memory
Double
Extra Large
34.2 GiB 13 4 (with 3.25
ECUs each)
840 GiB (1 x
840 GiB)
64-bit High Yes m2.2xlarge
High-
Memory
Extra Large
17.1 GiB 6.5 2 (with 3.25
ECUs each)
410 GiB (1 x
410 GiB)
64-bit Moderate Yes m2.xlarge
High-
Memory
Quadruple
Extra Large
68.4 GiB 26 8 (with 3.25
ECUs each)
1680 GiB (2
x 840 GiB)
64-bit High Yes m2.4xlarge

39. Cluster
Name Memory Compute
Units
Virtual Cores Instance Store
Volumes
Architecture I/O Performance Available for
Spot
Instance
API Name
Cluster
Compute Eight
Extra Large
60.5 GiB 88 16 (2 x Intel Xeon E5-
2670, eight-core with
hyperthread)
3360 GiB (4 x
840 GiB)
64-bit Very high (10
Gbps Ethernet)
Yes cc2.8xlarge
Cluster
Compute
Quadruple
Extra Large
22.5 GiB 33.5 8 (2 x Intel Xeon
X5570, quad-core with
hyperthread)
1690 GiB (2 x
840 GiB)
64-bit Very high (10
Gbps Ethernet)
Yes cc1.4xlarge
High-Memory Cluster
High-Memory
Cluster Eight
Extra Large
244 GiB 88 16 (2 x Intel Xeon E5-
2670, eight-core)
240 GiB (2 x
120 GiB SSD)
64-bit Very high (10
Gbps Ethernet)
Yes cr1.8xlarge
Cluster GPU
Cluster GPU
Quadruple
Extra Large**
22.5 GiB (see
note after this
table)
33.5 8 (2 x Intel Xeon
X5570, quad-core with
hyper thread), plus 2
NVIDIA Tesla M2050
GPUs
1680 GiB (2 x
840 GiB)
64-bit Very high (10
Gbps Ethernet)
Yes cg1.4xlarge

40. 1:Many Relationship Between AMIs and Instances
Amazon
Machine Image
(AMI)
Instance

41. Server/Instance States
Traditional Hardware-Based Server States
Amazon
EC2 AMI
Amazon EC2 Instance States

42. Physical Interface
Firewall
Hypervisor
Small
Large
…
Small
A Physical Host Has Multiple VMs
Virtual Machines
Physical Host

43. Auto Scaling
• Scale your Amazon EC2 capacity automatically
• Well suited for applications that experience variability in usage
• Available at no additional charge

44. Auto Scaling
• Automatically Scale Server Farms
– Scale up and down
– (Re)Balance Across AZs
– Add/Remove from ELB if applicable
• Set a Thermostat
– Don’t manage the furnace burners

45. Types of scaling
• Manual
– Send an API call or use CLI to launch/terminate instances
– Only need to specify capacity change (+/-)
• By Schedule
– Scale up/down based on date and time
• By Policy
– Scale in response to changing conditions, based on user configured real-
time monitoring and alerts
• Automatic Rebalance
– Instances are automatically launched/terminated to ensure the application is
balanced across multiple AZs

46. Auto Scaling
myASG
Launch
Configuration
ami-0535d66c
sa-east-1a sa-east-1b
myELB
myLC

47. Auto Scaling
Launch
Configuration
ami-0535d66c
sa-east-1a sa-east-1b
myELB
Trigger: CPULoad
Measure (M) : Average CPUUtilization
Scale-out by 1 if M > 80% for 5 minutes
Scale-in by 1 if M < 40% for 20 minutes
myASG
myLC

48. Auto Scaling
Launch
Configuration
ami-0535d66c
sa-east-1a sa-east-1b
myELB
Trigger: CPULoad
Measure (M) : Average CPUUtilization
Scale-out by 1 if M > 80% for 5 minutes
Scale-in by 1 if M < 40% for 20 minutes
myASG
myLC
as-create-auto-scaling-group myASG –launch-configuration myLC –availability-zones sa-east-1a, sa-east-1b
–min-size 1 –max-size 10 –desired-capacity 3 –load-balancers myELB
In this case the auto scaling group
knowsabout the ELB myELB because it
was identified when the auto scaling
group was created so the new
instance is added to the ELB myELB
pool

49. Anti-Pattern: Vertical
Vertical scaling (more CPU, memory, etc.) will eventually run out of room.
Auto Scaling Patterns

50. Anti-Pattern: Vertical
Vertical scaling (more CPU, memory, etc.) will eventually run out of room.
Auto Scaling Patterns

51. Pattern: Horizontal
Add and remove instances as needed
Auto Scaling Patterns

52. Pattern: Horizontal
Add and remove instances as needed
Auto Scaling Patterns

53. • O Magazine Luiza é uma das maiores
redes varejistas com foco em bens
duráveis e grande presença nas classes
populares do Brasil. Conta com uma base
de 30 milhões de clientes cadastrados,
sendo 30% deles ativos.
• Há 2 anos criamos o luizalabs - braço de
Pesquisa e Desenvolvimento do Magazine
Luiza. Com o objetivo de criar novos
produtos e servir de base de conhecimento
para novas tecnologias.
“Plataforma mobile que suporta o dobro do volume de acessos,
gera quase 50% mais conversões - com apenas 15% do valor
gasto anteriormente”
“Escolhemos a AWS por
4 motivos:
Performance,
Flexibilidade, Segurança
e Preço.”
- André Fatala

54. O Desafio
• Substituir a primeira versão do site mobile do
Magazine Luiza lançado em Dezembro de 2011 e
hospedado em uma parceira da produtora.
• Resolvemos desenvolver (in house) a segunda
versão do site mobile, objetivando uma aplicação
mais clean, facilidade de navegação e,
principalmente, uma navegação mais rápida
devido aos problemas de conexão que
enfrentamos com operadoras no Brasil.
• Escalabilidade - já que temos um canal que
cresceu em média 98% comparado ao mesmo
período do ano passado.

55. Sobre o Papel da AWS e Benefícios
alcançados
• Reduzimos a despesa com hospedagem
em 85%
• Carregamento de páginas 3x mais rápida
que originou uma taxa de rejeição 53%
menor e um aumento na taxa de
conversão de 36%
• A infraestrutura da AWS com seus recursos
programáveis, permitiu que a nova
plataforma fosse desenvolvida e colocada
em produção em apenas 1 mês

56. Helping Enterprises with Hybrid IT Architectures
Most enterprises will run a hybrid IT architecture
Some workloads will run on-premises
Some workloads will run in the cloud
Goal is to make management & integration easy
Workloads can be migrated back and forth

57. Our “Hybrid” Focus: Tools to Support Hybrid IT
Architectures
Private Connections
Workload Migrations
Access Control Integration
Work with Existing
Management Tools
On-Premises Apps
Your Data Centers
Cloud Apps

58. Active Directory
VMware Images
Network Configuration
Your Data
Your On-Premises Apps
Your Data Centers
Users & Access Rules
VM Import/Export
Your Private Network
Our Storage
Your Cloud Apps
Direct Connect
VPC
IAM
Storage Gateway
Our “Hybrid” Focus: Tools to Support Hybrid IT Architectures

59. Cloud benefits
Run cloud, run

60. Amazon RDS
RDS is a fully managed relational database service that is
simple to deploy, easy to scale, reliable and cost-effective
Ease of Deployment and Patching
Push Button Scalability
Choice of DB Engines and Application Compatibility
Automated Backups and Disaster Recovery
User Snapshots and Cloning
Monitoring and Automatic Host Replacement

61. Petabyte-Scale Data Warehousing
Feature Details
Optimized for Data
Warehousing
Redshift uses a variety of innovations to
obtain very high query performance on
datasets ranging in size from hundreds of
gigabytes to a petabyte or more.
Scalable Easily scale the number of nodes in your data
warehouse up or down as your performance
or capacity needs change
Fault tolerant Data replicated across Availability Zones.
Monitoring Integrated to CloudWatch.
Secure Encrypt data in transit and at rest. Can also
be run in VPC to isolate your data warehouse
cluster.
S3 intergration Loads data in parallel to each node from S3.
Elastic MapReduce Integrates with ERM via Data Pipeline.

62. On-prem
ises
On-prem
ises
AW
S Cloud
AW
S Cloud
DISASTER RECOVERY
FOR LOCAL APPLICATIONS
Amazon
EC2
Amazon
VPC
Amazon
S3
AWS Storage Gateway
AW
S
Reference
Architectures
Amazon
EBS
Amazon
S3
Amazon
VPC
AW
S
Storage
Gateway
VPC
GatewayVPC
Gateway
Corporate
User
Database
Server
(Recovery)Database
Server
(Recovery)
Application
Server
(Recovery)Application
Server
(Recovery)
4
3
Internet
Gateway
Snapshots
AMIs
Files
Storage Volumes
Secure
Connection
Secure
Connection
Amazon
EC2
Amazon
EC2
2
1
5
OracleS ecure
Backups
Secure
Connection
Secure
Connection
Application
Server
(Production)Application
Server
(Production)
Database
Server
(Production)Database
Server
(Production)
Corporate
DataCenter
DataRestoration
Amazon
EBS
Amazon
EBS
5
6

63. Michel Pereira
Hybrid Architecture: Integrating your Data Center with AWS
michelp@amazon.com