The document introduces Amazon EC2, MPI, and provides an example MPI "Hello World" program. It describes how EC2 allows users to rent virtual machines from a variety of configurations. It also explains basic MPI concepts like point-to-point communication and collective operations for exchanging data between processes.

1. 1: Introduction to Amazon EC2 and MPI
Zubair Nabi
zubair.nabi@itu.edu.pk
April 17, 2013
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 1 / 13

2. Outline
1 Amazon EC2
2 Message Passing Interface
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 2 / 13

3. Outline
1 Amazon EC2
2 Message Passing Interface
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 3 / 13

4. Introduction
Elastic Compute Cloud (EC2) allows you to rent out virtual machines
and run them atop Amazon’s physical datacenters
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 4 / 13

5. Introduction
Elastic Compute Cloud (EC2) allows you to rent out virtual machines
and run them atop Amazon’s physical datacenters
Possible to choose from a number of machine configurations
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 4 / 13

6. Introduction
Elastic Compute Cloud (EC2) allows you to rent out virtual machines
and run them atop Amazon’s physical datacenters
Possible to choose from a number of machine configurations
Vary in terms of processing power, memory configuration, and disk
space
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 4 / 13

7. Introduction
Elastic Compute Cloud (EC2) allows you to rent out virtual machines
and run them atop Amazon’s physical datacenters
Possible to choose from a number of machine configurations
Vary in terms of processing power, memory configuration, and disk
space
Amazon Machine Image (AMI), which is a silo of preconfigured
operating system stack and application software, is run atop these VMs
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 4 / 13

8. Introduction
Elastic Compute Cloud (EC2) allows you to rent out virtual machines
and run them atop Amazon’s physical datacenters
Possible to choose from a number of machine configurations
Vary in terms of processing power, memory configuration, and disk
space
Amazon Machine Image (AMI), which is a silo of preconfigured
operating system stack and application software, is run atop these VMs
Public AMIs work out-of-the-box with different application stacks, such
as those designed for LAMP, Drupal, etc.
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 4 / 13

9. Instances
Instances vary from Small to High-Memory Quadruple Extra Large and
from Cluster GPU to High I/O Quadruple Extra Large
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 5 / 13

10. Instances
Instances vary from Small to High-Memory Quadruple Extra Large and
from Cluster GPU to High I/O Quadruple Extra Large
M1 Small Instance:
1.7 GB memory
1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
160 GB instance storage
32-bit or 64-bit platform
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 5 / 13

11. Instances
Instances vary from Small to High-Memory Quadruple Extra Large and
from Cluster GPU to High I/O Quadruple Extra Large
M1 Small Instance:
1.7 GB memory
1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
160 GB instance storage
32-bit or 64-bit platform
High-Memory Quadruple Extra Large Instance:
68.4 GB memory
26 EC2 Compute Units (8 virtual core with 3.25 EC2 Compute Units each)
1690 GB instance storage
64-bit platform
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 5 / 13

12. Instances (2)
Cluster GPU Quadruple Extra Large Instance:
22 GB memory
33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core)
2 x NVIDIA Tesla “Fermi” M2050 GPUs
1690 GB instance storage
64-bit platform
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 6 / 13

13. Instances (2)
Cluster GPU Quadruple Extra Large Instance:
22 GB memory
33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core)
2 x NVIDIA Tesla “Fermi” M2050 GPUs
1690 GB instance storage
64-bit platform
High I/O Quadruple Extra Large Instance:
60.5 GB memory
35 EC2 Compute Units (16 virtual cores)
2 SSD-based volumes each with 1024 GB of instance storage
64-bit platform
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 6 / 13

14. Storage
Normal instances do not provide persistent storage
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 7 / 13

15. Storage
Normal instances do not provide persistent storage
Changes made to a VM are lost upon termination
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 7 / 13

16. Storage
Normal instances do not provide persistent storage
Changes made to a VM are lost upon termination
To remedy this, Elastic Block Store (EBS) can be used
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 7 / 13

17. Storage
Normal instances do not provide persistent storage
Changes made to a VM are lost upon termination
To remedy this, Elastic Block Store (EBS) can be used
Persistent virtual drives that give the illusion of network-attached
storage
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 7 / 13

18. S3
Storage with a simple web services interface
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

19. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

20. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
The unit of storage allocation is a bucket which is a container for
objects
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

21. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
The unit of storage allocation is a bucket which is a container for
objects
Objects consist of object data and some metadata (key/value pairs)
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

22. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
The unit of storage allocation is a bucket which is a container for
objects
Objects consist of object data and some metadata (key/value pairs)
Each object has a unique identifier, a key, within a bucket
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

23. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
The unit of storage allocation is a bucket which is a container for
objects
Objects consist of object data and some metadata (key/value pairs)
Each object has a unique identifier, a key, within a bucket
Updates are atomic at the key level
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

24. S3
Storage with a simple web services interface
Web interface used to store and retrieve data, at any time, from
anywhere on the web
The unit of storage allocation is a bucket which is a container for
objects
Objects consist of object data and some metadata (key/value pairs)
Each object has a unique identifier, a key, within a bucket
Updates are atomic at the key level
Automatic replication across multiple servers
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 8 / 13

25. Outline
1 Amazon EC2
2 Message Passing Interface
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 9 / 13

26. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

27. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

28. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

29. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Synchronous, asynchronous, and buffered variants
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

30. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Synchronous, asynchronous, and buffered variants
Cartesian or graph-centric logical process topologies
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

31. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Synchronous, asynchronous, and buffered variants
Cartesian or graph-centric logical process topologies
Exchanging data between process pairs
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

32. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Synchronous, asynchronous, and buffered variants
Cartesian or graph-centric logical process topologies
Exchanging data between process pairs
Aggregating partial results of computations
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

33. MPI
Language-independent communications protocol to build parallel
applications within and across machines
Interface to define virtual topology, synchronization, and
communication between processes
Point-to-point rendezvous-type send/receive operations
Synchronous, asynchronous, and buffered variants
Cartesian or graph-centric logical process topologies
Exchanging data between process pairs
Aggregating partial results of computations
Synchronizing nodes
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 10 / 13

34. Features
Used for distributed memory parallelism (shared-nothing systems)
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 11 / 13

35. Features
Used for distributed memory parallelism (shared-nothing systems)
Highly efficient: Can be used for many different models of computation,
therefore can be used with many different applications
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 11 / 13

36. Features
Used for distributed memory parallelism (shared-nothing systems)
Highly efficient: Can be used for many different models of computation,
therefore can be used with many different applications
Used for SPMD-style coding
Single Program, Multiple Data
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 11 / 13

37. Features
Used for distributed memory parallelism (shared-nothing systems)
Highly efficient: Can be used for many different models of computation,
therefore can be used with many different applications
Used for SPMD-style coding
Single Program, Multiple Data
Low-level therefore highly efficient
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 11 / 13

38. Example: Hello World!
1 #include <stdio.h >
2 #include mpi.h
< >
3
4 int main (int argc , char ∗argv [])
5 {
6 int rank , size;
7
8 MPI_Init (&argc , &argv );
9 MPI_Comm_rank ( MPI_COMM_WORLD , &rank );
10 MPI_Comm_size ( MPI_COMM_WORLD , &size );
11 printf (" Hello world on process %d of total %dn", rank , size );
12 MPI_Finalize ();
13 return 0;
14 }
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 12 / 13

39. References
Running a website on Amazon EC2:
http://www.diginmotion.com/faq/
108-running-a-website-on-amazon-ec2.pdf
Zubair Nabi 1: Introduction to Amazon EC2 and MPI April 17, 2013 13 / 13