Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon Elastic Computing 2


Published on

This presentation accompanied a practical demonstration of Amazon's Elastic Computing services to CNET students at the University of Plymouth on 16/03/2010.

The practical demonstration involved an obviously parallel problem split on 5 Medium size AMIs. The problem was the calculation of the Clustering Coefficient and the Mean Path Length (Based on the original work done by Watts and Strogatz) for large networks. The code was written in Python taking advantage of the scipy, pyparallel and networkx toolkits

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Amazon Elastic Computing 2

  1. 1. Amazon Elastic Computing 2<br />AthanasiosAnastasiou<br />Signal Processing And Multimedia Communications Research Group<br />University of Plymouth - UK<br />
  2. 2. Topics<br />How Did We Get Here?<br />Enabling Technologies<br />Amazon Elastic Computing<br />Why?<br />What?<br />How?<br />A Quick Demonstration<br />Exploring Complex Networks<br />Further Reading & Resources<br />
  3. 3. How Did We Get Here?(Enabling Technologies)<br />1939 The (Modern) Computer Is Born<br />Almost instantly people start thinking about connecting many units (CPUs) together…<br />1960 The (Modern) Network Is Born<br />1964 The ‘Virtual Machine’ Is Born<br />1967 Paper on Amdahl’s Law<br />1970 The Internet Is Born (ARPANET)<br />(Modern) Distributed Computing Is Born<br />1975 The Personal Computer Is Born<br />Mass production of CPUs!!!<br />1988 SoftPC Is Released<br />
  4. 4. How Did We Get Here?(Enabling Technologies)<br />1990 The World Wide Web Is Born<br />A worldwide network of computers…Hmmm<br />Computer Clusters (Local or over the internet)<br />1991 Linux Is Born<br />1998 VMWare patents its virtualisation techniques<br />2002 GRID Computing<br />Bridging together a variety of technologies into ONE system.<br />2005  Today<br /> Cloud Computing<br />Resources (Virtual Computers And Storage Devices) are remotely accessible on demand by some other system over a network (the internet)<br />
  5. 5. Amazon Elastic ComputingWhy?<br />On Demand Remote Access To Resources<br />Computational<br />Rent access to computer(s)<br />Storage<br />Rent storage space<br />Easy, Cheap, Available<br />Loose Restrictions<br />Server instances, Databases, Bandwidth etc<br />By Itself An Enabling Technology To:<br />Commercial Projects<br />Scientific Projects<br />
  6. 6. AmazonElasticComputingWhat? (1/3)<br />Amazon<br />Online Enterprise<br />Elastic<br />Claiming Resources According To Your Needs<br />Computing<br />CPUs<br />Computational Time<br />What About Storage?<br />Amazon Cloud Storage (S3)<br />Create Disks<br />Mount them on your filesystem<br />Treat them like any other disk space<br />Amazon Elastic Computing Offers Just The Infrastructure<br />
  7. 7. Amazon Elastic ComputingWhat? (2/3)<br />
  8. 8. Amazon Elastic ComputingWhat? (3/3)<br />Amazon Elastic Computing Offers Just The Infrastructure<br />User Registration<br />Billing<br />User<br />Manage AMIs<br />Manage I.Ps<br />Manage Storage<br />Store AMIs<br />Services<br />CloudWatch<br />Auto Scaling<br />Load Balancing<br />
  9. 9. Amazon Elastic ComputingWhat…(are the prices like?)<br /><ul><li> 1 ECU = 1 CPU @ 1.0-1.2 GHz (2007 Xeon or Opteron)
  10. 10. More and up to date information about pricing and instance availability are always available from here and here</li></li></ul><li>Amazon Elastic ComputingHow?<br />Initial Registration<br />Step-By-Step Instructions<br />Amazon Web Services Management Console (AWS MS)<br />High Level Control To All Your Services<br />Access To The Actual Instances<br />SSH <br />Or Putty on Windows<br />SCP<br />Or Winscp on Windows<br />
  11. 11. Amazon Elastic ComputingHow?<br />
  12. 12. Amazon Elastic ComputingHow?<br />
  13. 13. Amazon Elastic ComputingHow?<br />
  14. 14. Amazon Elastic ComputingHow?<br />
  15. 15. Any Questions So Far?<br />
  16. 16. OK, Let’s Do Something With It!!!<br />Time Consuming Tasks<br />3D Rendering<br />Computational Fluid Dynamics<br />Simulation<br />Search Through A Large / Huge Domain<br />
  17. 17. About The Demonstration<br />Search Through A Large Domain<br />Networks<br />Duncan Watts, Steven Strogatz, 1998, Collective Dynamics of ‘Small World’ Networks<br />Networks<br />Abstract construction with many practical applications<br />Nodes<br />Edges<br />Structure<br />Lattice<br />Random<br />Function<br />Structure affects the emergent functionality<br />What if a network is just a little bit random?<br />
  18. 18. Exploring Complex Networks<br />Lattice<br />Random<br />Small World<br />Rewiring Probability (p)<br />Different p values lead to networks with varying structures.<br />How can we characterise these networks?<br />
  19. 19. Exploring Complex Networks<br /><ul><li>For networks with N nodes, where each node is connected to K others
  20. 20. Rewire Each Edge With Probability P
  21. 21. Calculate Some Metrics Of Structure
  22. 22. The Clustering Coefficient (C)
  23. 23. The Mean Path Length (L)
  24. 24. Perform this step many times and obtain an average value</li></ul>B<br />A<br />Clustering Coefficient:<br />Path Length:<br />
  25. 25. Exploring Complex Networks<br />N=128, K=12, P=0.5<br />
  26. 26. Exploring Complex Networks<br />The Internet<br />The Opte Project ( <br />
  27. 27. Exploring Complex Networks<br />Networks with the ‘Small World’ property are everywhere…<br />Friendships<br />The Internet<br />The Brain<br />
  28. 28. Exploring Complex Networks<br />Let’s try and replicate Watts & Strogatz’s results!<br />Based on Python<br />Scipy<br />PyParallel<br />Networkx<br />Amazon Elastic Computing<br />A custom AMI based on Fedora<br />All necessary software already installed<br />1 Small Instance (Acting as a “Coordinator”)<br />5 Medium Instances (Acting as “Workers”)<br />
  29. 29. Further Reading & Resources<br />The timeline was created with material from the following sources<br />Computer History, Network History, Virtualisation Technology History, Linux Development Timeline, Super Computers Timeline<br />Some Noteworthy Parallel Processing Projects Where YOU can take part!<br />SETI@home<br />Folding@Home<br />Some Noteworthy Virtualisation Software<br />The XenHupervisor (and cloud computing infrastructure)<br />Oracle’s Virtualbox<br />
  30. 30. Further Reading & Resources<br />Amazon Web Services<br />A huge resource about amazon’s cloud computing infrastructure<br />Google App Engine<br />Specifically targeted to web applications.<br />Or, build your own cloud!<br />With Ubuntulinux<br />Python<br />The official web page<br />Scipy<br />An example of a service that “integrates” with Amazon Cloud Computing<br />Perhaps the natural evolution of Cloud Computing<br />PiCloud<br />
  31. 31. Further Reading & Resources<br />Mapping The Internet<br />For some HUGE graph datasets!<br />The Internet Mapping Project<br />The Opte Project<br />Books<br />SelimAkl, Parallel Computation: Models And Methods<br />Behrooz, Pahrami, Introduction To Parallel Processing<br />J. Rittinghouse & J. Ransome, Cloud Computing Implementation, Management and Security<br />
  32. 32. Thank You<br /><ul><li>Parallel Computing has enabled scientists to advance into unchartered territories and uncover patterns and information that is hidden deep into huge and complex datae. No matter how powerful individual computers will become…There will always be a necessity (or temptation) to connect them in parallel!</li></ul>AthanasiosAnastasiou<br /><br />