Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling HPC applications in EC2
with Elastic Fab...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
HPC applications in AWS
What is EFA?
Gett...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Related breakouts
Wednesday, Nov 28
High Perform...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Weather simulation
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0
20
40
60
80
100
120
140
160
0
2000
4000
6000
8...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trek applications for engineering:
• Computation...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fluid dynamics – Ansys Fluent
C4.8xlarge instanc...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC in aerospace
Boom leverages Rescale and AWS ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Children’s Hospital of Philadelphia and Edico Ge...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EBS Volumes
Enhanced Networking
Hardware
Quick A...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC software stack in Amazon EC2
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC software stack in Amazon EC2
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC network performance
0
500
1000
1500
2000
250...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC software stack with EFA
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
HPC software stack with EFA
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EBS Volumes
Enhanced Networking
Hardware
Introdu...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EFA getting started
• Supported platforms
• C5n....
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EFA constraints
• Subnet-local communication
• M...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s build an EFA-enabled cluster
% aws ec2 run...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What we just built…
Availability Zone #1
Subnet ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s build an EFA-enabled cluster
% aws ec2 run...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
And verify drivers…
% lsmod | grep efa
efa 81920...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
And verify MPI
% ompi_info | grep 'mtl: ofi'
MCA...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do I use EFA with…
AWS CloudFormation: Launc...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating a launch template with EFA
{
"NetworkIn...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we used to write HPC applications
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Libfabric changes the picture
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Libfabric Components
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EFA and libfabric: endpoints
Two native endpoint...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EFA native modes
• RDM
• Reliable, unordered dat...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Utility endpoint type
• RxR: build libfabrics in...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scalable Reliable Datagram (SRD)
• New protocol ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SRD link failure handling
0
5000
10000
15000
200...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F.A.Q.s
• When will it reach general availabilit...
Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Brian Barrett
bbarrett@amazon.com
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
[NEW LAUNCH!] Scaling Tightly-coupled HPC workloads on HPC with Elastic Fabric Adapter and High Bandwidth (Network Optimiz...
Upcoming SlideShare
Loading in …5
×

[NEW LAUNCH!] Scaling Tightly-coupled HPC workloads on HPC with Elastic Fabric Adapter and High Bandwidth (Network Optimized) EC2 Instances. (ENT360) - AWS re:Invent 2018

4,936 views

Published on

Elastic Fabric Adapter (EFA) is a network interface for Amazon EC2 instances that enables customers to run HPC applications requiring high levels of inter-instance communications, like computational fluid dynamics, weather modeling, and reservoir simulation, at scale on AWS. It uses a custom-built operating system bypass technique to enhance the performance of inter-instance communications, which is critical to scaling HPC applications. With EFA, HPC applications using popular HPC technologies like Message Passing Interface (MPI) can scale to thousands of CPU cores. Get a deep dive on EFA and learn how to use EFA to enhance application performance for your HPC workloads.

  • Want to earn $4000/m? Of course you do. Learn how when you join today! ◆◆◆ http://ishbv.com/ezpayjobs/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

[NEW LAUNCH!] Scaling Tightly-coupled HPC workloads on HPC with Elastic Fabric Adapter and High Bandwidth (Network Optimized) EC2 Instances. (ENT360) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scaling HPC applications in EC2 with Elastic Fabric Adapter Brian Barrett Principal Engineer AWS
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda HPC applications in AWS What is EFA? Getting started with EFA EFA tech deep-dive
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Related breakouts Wednesday, Nov 28 High Performance Computing on AWS 3:15 – 4:15 | Aria East, Piazza Level, Orovada 2 Wednesday, Nov 28 Running High Performance Computing Workloads in the Cloud 4:00 – 5:00 | Aria West, Level 3, Starvine 3, Table 8 Wednesday, Nov 28 Deploying a Burstable and Event-Driven HPC Cluster on AWS 1:00 – 2:00 | Aria West, Level 3, Starvine 10, Table 6
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Weather simulation
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 20 40 60 80 100 120 140 160 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 0 50 100 150 200 250 300 350 ScaleUp Time(s) Cores c4.8xlarge Time c4.8xlarge Scaleup Structural simulation
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Trek applications for engineering: • Computational Fluid Dynamics • Star-CCM+ and HEEDS software Cloud for product design and engineering Simulations for bicycle design: • Execute multiple simulations in parallel • Fully explore the design space to make informed decisions about drafting techniques related to competitive bicycling
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fluid dynamics – Ansys Fluent C4.8xlarge instance type 140M cell model F1 car CFD benchmark
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC in aerospace Boom leverages Rescale and AWS to enable supersonic travel • Simulated vortex lift with 200M cell models on 512+ cores • Increased simulation throughput: 100 jobs in parallel with 6x speedup per job → 600x speedup • Elastic HPC capacity and pay-as- you-go AWS clusters allow business agility & ability to scale “Rescale’s ScaleX cloud platform is a game-changer for engineering. It gives Boom computing resources comparable to building a large on- premise HPC center. Rescale lets us move fast with minimal capital spending and resources overhead.” Josh Krall CTO & Co-Founder
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Children’s Hospital of Philadelphia and Edico Genome Achieve Fastest-Ever Analysis of 1,000 Genomes Orlando, Fla., Oct 19, 2018 – The Children’s Hospital of Philadelphia (CHOP) and Edico Genome today set a new scientific world standard in rapidly processing whole human genomes into data files usable for researchers aiming to bring precision medicine into mainstream clinical practice. Utilizing Edico Genome’s DRAGENTM Genome Pipeline, deployed on 1,000 Amazon EC2 F1 instances on the Amazon Web Services (AWS) Cloud, 1,000 pediatric genomes were processed in two hours and 25 minutes. Genomics processing on FPGA
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EBS Volumes Enhanced Networking Hardware Quick Amazon Elastic Compute Cloud (Amazon EC2) review c5n.18xlarge Software NVMe ENA
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC software stack in Amazon EC2
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC software stack in Amazon EC2
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC network performance 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 EC2 MPI multi-stream bandwidth Series1 Series2 Series3 Series4 0 10 20 30 40 50 60 1 2 3 4 EC2 MPI Latency
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC software stack with EFA
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. HPC software stack with EFA
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EBS Volumes Enhanced Networking Hardware Introducing EFA c5n.18xlarge Software NVMe ENAEFA
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 2000 4000 6000 8000 10000 12000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 EC2 MPI multi-stream bandwidth Series1 Series2 Series3 Series4 Series5 0 10 20 30 40 50 60 1 2 3 4 5 EC2 MPI Latency HPC network performance with EFA
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EFA getting started • Supported platforms • C5n.18xlarge, C5n.9xlarge, P3dn.24xlarge • EFA Kernel module • Upstream in progress • https://github.com/amzn/amzn-drivers • Libfabric Network Stack • AWS-custom version for first half 2019 • MPI Implementation or NCCL • Open MPI 3.1.3 or later or NCCL 2.3.8 or later • Intel MPI and MPICH in development See https://aws.amazon.com/hpc/ for more details
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EFA constraints • Subnet-local communication • Must have both an “allow all traffic within security group” ingress and egress rule • 1 EFA ENI per instance • EFA ENIs can only be added at instance launch or to stopped instance
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s build an EFA-enabled cluster % aws ec2 run-instances --count=4 --region us-east-1 --image-id ami-ABCD --instance-type c5n.18xlarge --placement GroupName=ABCD --network-interfaces DeleteOnTermination=true,DeviceIndex=0, SubnetId=subnet-ABCD,InterfaceType=efa --security-group-ids sg-ABCD
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What we just built… Availability Zone #1 Subnet ABCD
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s build an EFA-enabled cluster % aws ec2 run-instances --count=4 --region us-east-1 --image-id ami-ABCD --instance-type c5n.18xlarge --placement GroupName=ABCD --network-interfaces DeleteOnTermination=true,DeviceIndex=0, SubnetId=subnet-ABCD,InterfaceType=efa --security-group-ids sg-ABCD 🕰 % ssh ec2-user@ec2-1-2-3-4.compute-1.amazonaws.com % lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 00:06.0 Ethernet controller: Amazon.com, Inc. Device efa0
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. And verify drivers… % lsmod | grep efa efa 81920 0 ib_core 266240 2 efa,ib_uverbs % fi_info -p efa provider: efa fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-rdm version: 3.0 type: FI_EP_RDM protocol: FI_PROTO_EFA provider: efa fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-dgrm version: 3.0 type: FI_EP_DGRAM protocol: FI_PROTO_EFA provider: efa;ofi_rxr fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-rdm version: 1.0 type: FI_EP_RDM protocol: FI_PROTO_RXR
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. And verify MPI % ompi_info | grep 'mtl: ofi' MCA mtl: ofi (MCA v2.1.0, API v2.0.0, Component v3.1.3) % mpirun -np 2 -hostfile ~/h ./ring_c Process 0 sending 10 to 1, tag 201 (2 processes in ring) Process 0 sent to 1 Process 0 decremented value: 9 Process 0 decremented value: 8 Process 0 decremented value: 7 Process 0 decremented value: 6 Process 0 decremented value: 5 Process 0 decremented value: 4 Process 0 decremented value: 3 Process 0 decremented value: 2 Process 0 decremented value: 1 Process 0 decremented value: 0 Process 0 exiting Process 1 exiting
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do I use EFA with… AWS CloudFormation: Launch Templates Amazon EC2 Auto Scaling Groups: Launch Templates Spot/Spot Fleet: Launch Templates AWS Batch: Launch Templates Launch Templates: add InterfaceType : efa to the Network section
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Creating a launch template with EFA { "NetworkInterfaces": [{ "AssociatePublicIpAddress": false, "DeviceIndex": 0, "SubnetId": "subnet-ABCD", "InterfaceType" : "efa" }], "Placement " : { "GroupName": "ABCD" }, "ImageId": "ami-ABCD", "InstanceType": "c5n.18xlarge", "CpuOptions": { ”CoreCount": 36, "ThreadsPerCore": 1 } }
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How we used to write HPC applications
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Libfabric changes the picture
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Libfabric Components
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EFA and libfabric: endpoints Two native endpoint types: • RDM (Reliable DataGram) • DGRM (unreliable DataGRaM) One utility endpoint type: • RxR (RDM over RDM) EFA protocol custom to AWS % fi_info -p efa provider: efa fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-rdm version: 3.0 type: FI_EP_RDM protocol: FI_PROTO_EFA provider: efa fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-dgrm version: 3.0 type: FI_EP_DGRAM protocol: FI_PROTO_EFA provider: efa;ofi_rxr fabric: EFA-fe80::883:afff:fed3:776c domain: efa_0-rdm version: 1.0 type: FI_EP_RDM protocol: FI_PROTO_RXR
  35. 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EFA native modes • RDM • Reliable, unordered datagrams • ~8 KiB max message size • Send/receive interface, with no tag matching • Native multi-pathing; no “flow limit” • DGRAM • Unreliable, unordered datagrams • ~8 KiB max message size • Send/receive interface • Subject to same “flow limit” as TCP/IP and UDP/IP over ENA
  36. 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Utility endpoint type • RxR: build libfabrics interface over RDM • Completion ordered datagrams • tagged matching support (ie, MPI) • Max message size > system memory size • Large iovecs • RxR developed by AWS as part of EFA • Contributing back to Libfabric community shortly • Currently implemented to support MPI implementations • Future work includes supporting RMA and atomic transfer interfaces
  37. 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scalable Reliable Datagram (SRD) • New protocol designed for AWS’s unique datacenter network • Network aware multipath routing • Guaranteed delivery • Orders of magnitude lower tail latency • No ordering guarantees • Implemented as part of our 3rd generation Nitro chip • EFA exposes SRD as a reliable datagram interface
  38. 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SRD link failure handling 0 5000 10000 15000 20000 25000 0 1000 2000 3000 4000 5000 6000 7000 TCP Series1 0 5000 10000 15000 20000 25000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 SRD Series1
  39. 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. F.A.Q.s • When will it reach general availability? • First half 2019 • How do I sign up for the preview? • https://pages.awscloud.com/elastic-fabric-adapter-preview.html • What regions will EFA launch in? • Any region with C5n or P3dn support • What is your MPI latency? • Less than 15 𝜇s ½ RTT in placement group (osu_latency benchmark) • We will be constantly iterating behind the scenes to lower latency, including expanding .metal options
  40. 40. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Brian Barrett bbarrett@amazon.com
  41. 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×