Paul Watson Newcastle University, UK [email_address] An Introduction to Cloud-based Services
<ul><li>e.g. Amazon </li></ul>
Plan <ul><li>What is Cloud Computing? </li></ul><ul><li>Potential Advantages </li></ul><ul><li>Lessons from our own experi...
What is Cloud Computing?  <ul><li>“ .. a broad array of </li></ul><ul><li>web-based services aimed at  </li></ul><ul><li>a...
What’s New? <ul><li>illusion of  Infinite  computing resources  On Demand </li></ul><ul><li>no  up-front  commitment by us...
Example – Amazon Web Services <ul><li>Based on Xen VMs </li></ul><ul><ul><li>run any OS & software stack </li></ul></ul><u...
Why is this Important (I): Internal IT Problems  (slide by permission of Arjuna Technologies) Silos = Inflexibility
Why is this Important (II)? Time to put Ideas into action <ul><li>Research </li></ul><ul><li>Have good idea </li></ul><ul>...
Why is this a Good idea: using commercial clouds <ul><li>Have good idea </li></ul><ul><li>Grab nodes as needed from Cloud ...
Cloud Services Continuum  (based on Robert Anderson) Google Docs Google AppEngine Amazon EC2 & S3 http://et.cairene.net/20...
Example Lessons from CARMEN Project <ul><li>Design began in 2006 </li></ul><ul><ul><li>Commercial clouds not an option </l...
CARMEN Project <ul><li>UK EPSRC e-Science Pilot </li></ul><ul><li>£4M (2006-10) </li></ul><ul><li>20 Investigators </li></...
Industry & Associates
Research Challenge <ul><li>Understanding the brain is the greatest informatics challenge </li></ul><ul><li>Enormous implic...
Collecting the Evidence <ul><li>100,000 neuroscientists  generate huge quantities of data   </li></ul><ul><ul><li>molecula...
Epilepsy Exemplar Data analysis guides surgeon during operation Further analysis provides evidence WARNING! The next 2 Sli...
CARMEN <ul><li>enables sharing and collaborative exploitation of data, analysis code and expertise that are not physically...
CARMEN e-Science Requirements <ul><li>Store </li></ul><ul><ul><li>very large quantities of data (100TB+) </li></ul></ul><u...
Background: North East Regional e-Science Centre <ul><li>25 Research Projects across many domains: </li></ul><ul><ul><ul><...
Result: e-Science Central <ul><li>Integrated  Store-Analyse-Automate-Share infrastructure </li></ul><ul><li>Generic </li><...
e-Science Central e-Science Central <ul><li>Web based </li></ul><ul><li>Works anywhere </li></ul><ul><li>Controlled Sharin...
Science Cloud Architecture <ul><li>Data storage </li></ul><ul><li>and </li></ul><ul><li>analysis </li></ul>Access over Int...
Science Cloud Options  Cloud Infrastructure: Storage & Compute Science App 1 .... Science App n Cloud Infrastructure: Sto...
App .... Workflow Enactment Social Networking App API Security Processing e-Science Central Storage App Analysis Services ...
Editing and Running a Workflow on the Web
Viewing the output of Workflow Runs Workflow Result File
Viewing results
Blogs and links Communicating Results Linking to  results & workflows
What we learnt: Moving into a Cloud <ul><li>Moving existing technologies into a cloud can be difficult </li></ul><ul><ul><...
Raw Data Exploration with Signal Data Explorer
What we learnt : Scalability <ul><li>Clouds offer the potential for scalability </li></ul><ul><ul><li>grab compute power o...
Adaptive Dynamic Deployment with Dynasoar Adding Processors as you need them optimises resources and saves money in pay-as...
Microsoft Azure Cloud for e-Science Demo <ul><li>Recent experiments with Microsoft Azure Cloud </li></ul><ul><ul><li>runni...
 
Microsoft Azure Cloud Demo
When not to use Clouds? <ul><li>Large data transfers </li></ul><ul><ul><li>Time & Cost </li></ul></ul><ul><li>High Perform...
Create Private Cloud    (slides by permission of Arjuna Technologies)
Private Cloud Examples <ul><li>Eucalyptus </li></ul><ul><ul><li>Amazon API </li></ul></ul><ul><li>Private Cloud deployment...
Federating Private  & Public Clouds Dept A Dept B App1 App1 & 2 Public Cloud App1 Arjuna Agility Public Cloud e.g. Amazon ...
Dept A Dept B App1 App1 & 2 Public Cloud e.g. Amazon App1 Public Cloud e.g. FlexiScale App1 Arjuna Internal Cloud Arjuna A...
Summary <ul><li>Cloud computing can revolutionise e-science </li></ul><ul><ul><li>provide sustainable infrastructure </li>...
Upcoming SlideShare
Loading in …5
×

PPT

644 views
583 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
644
On SlideShare
0
From Embeds
0
Number of Embeds
46
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Mobile Devices – Software + Services App stores Current Grid Computing – job submission When to use - When not to use High Utilisation Large data transfers -- costs + Jim Gray fedex Predictability - cpu , io, network High performance cpu/bandwidth/latency Availability? Privacy? Will your software run in the cloud? possible? cost to port Skills? Capex -&gt; opex
  • The diagram to the right shows this continuum from infrastructure to platform to software.   Brief definitions of these parts are: Infrastructure includes provisioning of hardware or virtual computers on which one generally has control over the OS; therefore allowing the execution of arbitrary software. Platform indicates a higher-level environment for which developers write custom applications.  Generally the developer is accepting some restrictions on the type of software they can write in exchange for built-in application scalability.  Software (as a Service) indicates special-purpose software made available through the Internet.
  • Complex – 100 billion neurons; up to 10K connections each
  • Hi Paul, okay so the patient is usually worked up using scalp EEG and other imaging (MR) techniques (to ascertain if there is say a distinct lesion perhaps that is causing the pathological activity). If this does not lead to a conclusive answer they also carry out ECoG (electrocorticography). The neurophysiologist, records from the brain during surgery and then instructs the neurosurgeon as to what areas he thinks are pathological. The tissue is then removed, we receive it and we record from this tissue using even more refined processes ( ie recordings from single neurons, large groups of neurons etc etc) Mark Cunningham
  • Rest of talk explores these and their solns in more detail
  • Bus Phone
  • A start towards reproducibility
  • PPT

    1. 1. Paul Watson Newcastle University, UK [email_address] An Introduction to Cloud-based Services
    2. 2. <ul><li>e.g. Amazon </li></ul>
    3. 3. Plan <ul><li>What is Cloud Computing? </li></ul><ul><li>Potential Advantages </li></ul><ul><li>Lessons from our own experiences </li></ul><ul><li>Cloud Issues </li></ul>
    4. 4. What is Cloud Computing? <ul><li>“ .. a broad array of </li></ul><ul><li>web-based services aimed at </li></ul><ul><li>allowing users to obtain a wide range of functional capabilities </li></ul><ul><li>on a ‘pay-as-you-go’ basis </li></ul><ul><li>that previously required tremendous hardware/software investments </li></ul><ul><li>and professional skills to acquire.” </li></ul><ul><ul><li>Irving Wladawsky Berger </li></ul></ul>
    5. 5. What’s New? <ul><li>illusion of Infinite computing resources On Demand </li></ul><ul><li>no up-front commitment by users </li></ul><ul><li>Pay for use of resources on a short-term basis as needed </li></ul><ul><li>(from “Above the Clouds: A Berkeley View of Cloud Computing”) </li></ul>
    6. 6. Example – Amazon Web Services <ul><li>Based on Xen VMs </li></ul><ul><ul><li>run any OS & software stack </li></ul></ul><ul><li>CPU: 1.0Ghz x86 instance @ $0.10 /hour </li></ul><ul><li>Blob Storage @ $0.12 /GB month </li></ul><ul><li>External Data Transfer @ $0.10 /GB </li></ul><ul><li>Also queue, key store, block store, range of instances </li></ul>
    7. 7. Why is this Important (I): Internal IT Problems (slide by permission of Arjuna Technologies) Silos = Inflexibility
    8. 8. Why is this Important (II)? Time to put Ideas into action <ul><li>Research </li></ul><ul><li>Have good idea </li></ul><ul><li>Write proposal </li></ul><ul><li>Wait 6 months </li></ul><ul><li>If successful.. </li></ul><ul><li>Buy Computers </li></ul><ul><li>Install Computers </li></ul><ul><li>Start Work </li></ul><ul><li>Science Start-ups </li></ul><ul><li>Have good idea </li></ul><ul><li>Write Business Plan </li></ul><ul><li>Ask VCs to fund </li></ul><ul><li>If successful.. </li></ul><ul><li>Buy computers </li></ul><ul><li>Install Computers </li></ul><ul><li>Start Work </li></ul>
    9. 9. Why is this a Good idea: using commercial clouds <ul><li>Have good idea </li></ul><ul><li>Grab nodes as needed from Cloud provider </li></ul><ul><li>Start Work </li></ul><ul><li>Pay for what you used </li></ul>
    10. 10. Cloud Services Continuum (based on Robert Anderson) Google Docs Google AppEngine Amazon EC2 & S3 http://et.cairene.net/2008/07/03/cloud-services-continuum/ Microsoft Azure Salesforce.com Flexibility Complexity Platform (PaaS) Infrastructure (IaaS) Software (SaaS)
    11. 11. Example Lessons from CARMEN Project <ul><li>Design began in 2006 </li></ul><ul><ul><li>Commercial clouds not an option </li></ul></ul><ul><li>Designed own “private” cloud </li></ul><ul><li>Experimenting with Commercial Cloud </li></ul>
    12. 12. CARMEN Project <ul><li>UK EPSRC e-Science Pilot </li></ul><ul><li>£4M (2006-10) </li></ul><ul><li>20 Investigators </li></ul>Stirling St. Andrews Newcastle York Sheffield Cambridge Imperial Plymouth Warwick Leicester Manchester
    13. 13. Industry & Associates
    14. 14. Research Challenge <ul><li>Understanding the brain is the greatest informatics challenge </li></ul><ul><li>Enormous implications for science: </li></ul><ul><ul><li>Medicine </li></ul></ul><ul><ul><li>Biology </li></ul></ul><ul><ul><li>Computer Science </li></ul></ul>
    15. 15. Collecting the Evidence <ul><li>100,000 neuroscientists generate huge quantities of data </li></ul><ul><ul><li>molecular (genomic/proteomic) </li></ul></ul><ul><ul><li>neurophysiological (time-series activity) </li></ul></ul><ul><ul><li>anatomical (spatial) </li></ul></ul><ul><ul><li>behavioural </li></ul></ul>
    16. 16. Epilepsy Exemplar Data analysis guides surgeon during operation Further analysis provides evidence WARNING! The next 2 Slides show an exposed human brain
    17. 17. CARMEN <ul><li>enables sharing and collaborative exploitation of data, analysis code and expertise that are not physically collocated </li></ul>
    18. 18. CARMEN e-Science Requirements <ul><li>Store </li></ul><ul><ul><li>very large quantities of data (100TB+) </li></ul></ul><ul><li>Analyse </li></ul><ul><ul><li>suite of neuroinformatics services </li></ul></ul><ul><ul><li>support data intensive analysis </li></ul></ul><ul><li>Automate </li></ul><ul><ul><li>workflow </li></ul></ul><ul><li>Share </li></ul><ul><ul><li>under user-control </li></ul></ul>
    19. 19. Background: North East Regional e-Science Centre <ul><li>25 Research Projects across many domains: </li></ul><ul><ul><ul><li>Bioinformatics, Ageing & Health, Neuroscience, Chemical Engineering, Transport, Geomatics, Video Archives, Artistic Performance Analysis, Computer Performance Analysis,.... </li></ul></ul></ul><ul><li>Same key needs: </li></ul>
    20. 20. Result: e-Science Central <ul><li>Integrated Store-Analyse-Automate-Share infrastructure </li></ul><ul><li>Generic </li></ul><ul><ul><li>CARMEN neuroinformatics & chemistry as pilots </li></ul></ul>
    21. 21. e-Science Central e-Science Central <ul><li>Web based </li></ul><ul><li>Works anywhere </li></ul><ul><li>Controlled Sharing </li></ul><ul><li>Collaboration </li></ul><ul><li>Communities </li></ul><ul><li>Dynamic Resource </li></ul><ul><li>Allocation </li></ul><ul><li>Pay-as-you-Go* </li></ul>
    22. 22. Science Cloud Architecture <ul><li>Data storage </li></ul><ul><li>and </li></ul><ul><li>analysis </li></ul>Access over Internet (typically via browser) Upload data & services Run analyses
    23. 23. Science Cloud Options  Cloud Infrastructure: Storage & Compute Science App 1 .... Science App n Cloud Infrastructure: Storage & Compute Science Platform Science App 1 .... Science App n Users Service Developers
    24. 24. App .... Workflow Enactment Social Networking App API Security Processing e-Science Central Storage App Analysis Services Science Cloud Platform Cloud Infrastructure
    25. 25. Editing and Running a Workflow on the Web
    26. 26. Viewing the output of Workflow Runs Workflow Result File
    27. 27. Viewing results
    28. 28. Blogs and links Communicating Results Linking to results & workflows
    29. 29. What we learnt: Moving into a Cloud <ul><li>Moving existing technologies into a cloud can be difficult </li></ul><ul><ul><li>some can’t run in a Cloud at all </li></ul></ul>
    30. 30. Raw Data Exploration with Signal Data Explorer
    31. 31. What we learnt : Scalability <ul><li>Clouds offer the potential for scalability </li></ul><ul><ul><li>grab compute power only when needed </li></ul></ul><ul><li>Developers have to manage scalability </li></ul><ul><ul><li>for Infrastructure as a Service Clouds </li></ul></ul><ul><ul><li>scale up as well as down </li></ul></ul>
    32. 32. Adaptive Dynamic Deployment with Dynasoar Adding Processors as you need them optimises resources and saves money in pay-as-you-go clouds Commercial “pay-as-you-go” clouds would allow us to avoid this limit Ensure system can also release unwanted nodes
    33. 33. Microsoft Azure Cloud for e-Science Demo <ul><li>Recent experiments with Microsoft Azure Cloud </li></ul><ul><ul><li>running Chemical analyses </li></ul></ul><ul><ul><li>Silverlight App </li></ul></ul><ul><li>Thanks to: </li></ul><ul><li>- Paul Appleby & Team at the Microsoft Technology Centre, Reading </li></ul><ul><li>- & MS External Research e-Science Group </li></ul>
    34. 35. Microsoft Azure Cloud Demo
    35. 36. When not to use Clouds? <ul><li>Large data transfers </li></ul><ul><ul><li>Time & Cost </li></ul></ul><ul><li>High Performance </li></ul><ul><ul><li>cpu/io/network bandwidth/low latency </li></ul></ul><ul><li>Predictable Performance </li></ul><ul><li>Confidentiality </li></ul><ul><li>High Availability? </li></ul><ul><li>High Server Utilisation? </li></ul><ul><ul><li>private clouds better? </li></ul></ul>
    36. 37. Create Private Cloud (slides by permission of Arjuna Technologies)
    37. 38. Private Cloud Examples <ul><li>Eucalyptus </li></ul><ul><ul><li>Amazon API </li></ul></ul><ul><li>Private Cloud deployments of Microsoft Azure </li></ul><ul><li>Arjuna Agility </li></ul>
    38. 39. Federating Private & Public Clouds Dept A Dept B App1 App1 & 2 Public Cloud App1 Arjuna Agility Public Cloud e.g. Amazon Internal Cloud Service Agreement Service Agreement
    39. 40. Dept A Dept B App1 App1 & 2 Public Cloud e.g. Amazon App1 Public Cloud e.g. FlexiScale App1 Arjuna Internal Cloud Arjuna Agility
    40. 41. Summary <ul><li>Cloud computing can revolutionise e-science </li></ul><ul><ul><li>provide sustainable infrastructure </li></ul></ul><ul><ul><li>reduce time from idea to realisation </li></ul></ul><ul><li>Don’t underestimate complexity </li></ul><ul><ul><li>building scalable distributed systems is still hard </li></ul></ul><ul><ul><li>can Science Clouds help by lowering the hurdles? </li></ul></ul><ul><li>e-Science Central </li></ul><ul><ul><li>Store-Analyse-Automate-Share e-science platform </li></ul></ul><ul><ul><li>adding content from a range of domains </li></ul></ul><ul><ul><li>CARMEN is evaluating it for neuroinformatics </li></ul></ul>

    ×