Living On A Cloud, Dr Keith Marlow

1,233 views

Published on

From presentation "Gaining leverate without the costs

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,233
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Living On A Cloud, Dr Keith Marlow

  1. 1. Living on a Cloud Dr Keith Marlow Chief Architect, APAC Region [email_address]
  2. 2. Agenda <ul><li>What cloud computing actually is… </li></ul><ul><li>The anatomy of a cloud.. </li></ul><ul><li>What does it really do for you? </li></ul><ul><li>Being cheap and leveraged </li></ul><ul><li>Yahoo API’s and Web Services </li></ul><ul><li>The future </li></ul><ul><li>Questions… </li></ul>
  3. 3. What Cloud Computing actually is… <ul><li>Really ancient history – way back in the early 1990’s… </li></ul><ul><li>All ‘in-house’ </li></ul><ul><li>All risks yours to manage </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage
  4. 4. What Cloud Computing actually is… <ul><li>Then in the mid to late 90’s – The Internet … </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage The Internet Not Business Critical! Full of none business ‘stuff’
  5. 5. What Cloud Computing actually is… <ul><li>Around 2000, the Internet became critical. </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage The Internet Full of businesses & customers! Critical to your Business!
  6. 6. What Cloud Computing actually is… <ul><li>But now with cloud computing we have.. </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage HOSTED SERVICES YOUR DATA? The Internet
  7. 7. What Cloud Computing actually is… <ul><li>Cloud Computing is either: </li></ul><ul><ul><li>Remotely hosted data processing services, or </li></ul></ul><ul><ul><li>Remotely hosted web services </li></ul></ul><ul><li>Which are: </li></ul><ul><ul><li>A highly distributed and flexible computing environment </li></ul></ul><ul><ul><li>With high availability </li></ul></ul><ul><li>Basically, it’s a remote self-scaling computing ‘resource’ </li></ul><ul><ul><li>Fire and forget </li></ul></ul><ul><li>Thanks, in part to: </li></ul><ul><ul><li>Cheaper bandwidth and hardware </li></ul></ul><ul><ul><li>Much faster machines </li></ul></ul><ul><ul><li>Abstraction and standards </li></ul></ul><ul><ul><li>Businesses, the research community & Open Source </li></ul></ul>
  8. 8. The anatomy of a cloud.. <ul><li>1 ST case – The Processing Cloud (Hadoop) </li></ul>MANAGEMENT SERVICES PROCESSING CLOUD 1000’s of nodes & disks DATA I/O CLIENT API ACCOUNTING & AUDITING
  9. 9. How does Hadoop scale? Map/Reduce Input Map Map Map Map Transient Data Results Reduce Reduce Reduce Reduce Split into ‘ bits’ Process the ‘bits’ on each node Collate each ‘bin’ on each node Shuffle into ‘ bins’ Join it all together
  10. 10. Hadoop – what is it? <ul><li>Open Source Apache project - http:// hadoop.apache.org /core/ </li></ul><ul><li>Hadoop Core includes: </li></ul><ul><ul><li>Distributed File System - distributes data between nodes </li></ul></ul><ul><ul><li>Map/Reduce - distributes application </li></ul></ul><ul><li>Written in Java </li></ul><ul><li>Runs on </li></ul><ul><ul><li>Linux, Mac OS/X, Windows, and Solaris </li></ul></ul><ul><ul><li>Commodity hardware </li></ul></ul>
  11. 11. Hadoop – How do we use it? <ul><li>Example: Web Search </li></ul><ul><ul><li>BIG graph: 100 billion nodes and 1 trillion edges </li></ul></ul><ul><ul><li>Largest shuffle is 450 TB (or 643000 CD’s worth!) </li></ul></ul><ul><ul><li>Final output is 300 TB compressed </li></ul></ul><ul><ul><li>Runs on 10,000 cores </li></ul></ul><ul><ul><li>Written in C++ </li></ul></ul>
  12. 12. Hadoop – How do we use it? What 20,0000 nodes look like
  13. 13. Hadoop – Real life usage… <ul><li>NY TIMES </li></ul><ul><li>Needed offline conversion of public domain articles from 1851-1922. </li></ul><ul><li>Used Hadoop to convert scanned images to PDF </li></ul><ul><li>Ran 100 Amazon EC2 instances for around 24 hours </li></ul><ul><li>4 TB of input </li></ul><ul><li>1.5 TB of output </li></ul>Published 1892, copyright New York Times
  14. 14. Hadoop – Who else uses it? <ul><li>Amazon/A9 </li></ul><ul><li>Facebook </li></ul><ul><li>Google </li></ul><ul><li>IBM </li></ul><ul><li>Joost </li></ul><ul><li>Last.fm </li></ul><ul><li>New York Times </li></ul><ul><li>PowerSet (now Microsoft) </li></ul><ul><li>Quantcast </li></ul><ul><li>Veoh </li></ul><ul><li>Yahoo! </li></ul><ul><li>Basically proven to be fit for purpose </li></ul><ul><li>More information at: </li></ul><ul><ul><li>http:// developer.yahoo.net/blogs/hadoop / </li></ul></ul>
  15. 15. The anatomy of a cloud.. <ul><li>2 nd case – The Web Services Cloud </li></ul>MANAGEMENT SERVICES DATA RETENTION PROCESSING CLOUD DATA I/O CLIENT API ACCOUNTING & AUDITING
  16. 16. The anatomy of a cloud.. <ul><li>2 nd case – The Web Services & Application Cloud </li></ul>MANAGEMENT SERVICES DATA RETENTION DATA I/O CLIENT API ACCOUNTING & AUDITING PROCESSING GRID 1000’s of nodes PROCESSING CLOUD 1000’s of nodes
  17. 17. Being cheap and leveraged… <ul><li>Cloud Computing allows time of usage outsourcing </li></ul><ul><ul><li>Only pay for exactly what you use </li></ul></ul><ul><ul><li>Lower CAPEX costs, greater ROI </li></ul></ul><ul><ul><li>Its green too! </li></ul></ul><ul><li>Needs a different approach to systems design </li></ul><ul><ul><li>New views on data to use in cloud & privacy protection </li></ul></ul><ul><ul><li>Decoupling around remote API’s </li></ul></ul><ul><ul><li>Remote hosting SDK’s training </li></ul></ul>
  18. 18. Being cheap and leveraged… <ul><li>Use the services of Yahoo! to implement and improve your services and offerings </li></ul><ul><ul><li>Low (or nil) set up and operational costs </li></ul></ul><ul><li>Yahoo provides the following: </li></ul><ul><ul><li>API’s and Web Services </li></ul></ul><ul><ul><li>RSS content feeds </li></ul></ul><ul><ul><li>Developer kits and GUI libraries </li></ul></ul><ul><ul><li>BrowserPlus </li></ul></ul><ul><ul><li>YAP </li></ul></ul><ul><li>To you they all operate as ‘clouds’ </li></ul>
  19. 19. Yahoo API’s and Web Services <ul><li>Maps </li></ul><ul><ul><li>Include a map on a website or intranet </li></ul></ul><ul><li>GeoPlanet ™ </li></ul><ul><ul><li>Geocode any address into latitude, longitude and WoeID </li></ul></ul><ul><li>Mail </li></ul><ul><ul><li>Send/Read email, lists folders etc </li></ul></ul><ul><ul><li>Zimbra – completely hosted mail service </li></ul></ul><ul><li>BOSS – Build you own Search Service </li></ul><ul><li>Search Monkey – Enriched search results for your sites </li></ul><ul><li>OpenID – share user ID’s between sites </li></ul><ul><li>YAP – Yahoo Application Platform – going Open on Yahoo! </li></ul><ul><li>http:// developer.yahoo.com / </li></ul>
  20. 20. BOSS <ul><li>API’s into Yahoo! Search </li></ul><ul><ul><li>Unlimited queries a day </li></ul></ul><ul><ul><li>No restrictions on presentation </li></ul></ul><ul><ul><li>Re-ordering allowed </li></ul></ul><ul><ul><li>Blending of Proprietary and Yahoo! Search Content Allowed </li></ul></ul><ul><ul><li>White-Label </li></ul></ul><ul><li>http://developer.yahoo.com/search/boss/ </li></ul><ul><li>Use to implement your own site search! </li></ul>
  21. 21. Search Monkey http://developer.yahoo.com/searchmonkey/
  22. 22. YAP – Yahoo! Application Platform <ul><li>Allows you to write applications modules that potentially can ‘run’ in any Yahoo! Web site (or supporting 3 rd party web site) </li></ul><ul><ul><li>i.e. Yahoo! itself becomes a ‘cloud’  </li></ul></ul><ul><ul><li>The user selects modules from a gallery to ‘paint’ onto their page canvas. </li></ul></ul><ul><ul><li>We take care of blending the modules with the content of the site to make the final page. </li></ul></ul><ul><li>What does this mean to you?? </li></ul><ul><ul><li>Ability to put your dynamic products and services onto Yahoo! </li></ul></ul><ul><ul><li>Closer relationship with users/customers in general </li></ul></ul>
  23. 23. The Future of Cloud Computing <ul><li>Smaller/Bigger/Faster/Cheaper </li></ul><ul><li>Hosting ‘in the cloud’ will become the norm </li></ul><ul><ul><li>Easier & cheaper than not doing so. </li></ul></ul><ul><li>Sum of the parts will be greater than the whole </li></ul><ul><ul><li>More ‘on the fly’ services aggregation & customization </li></ul></ul><ul><li>The user will ‘combine’ services to meet their needs. </li></ul><ul><li>The desktop PC/TV/Mobile will become the ‘presentation & personalization gateway’ into the Internet Cloud </li></ul>
  24. 24. Questions??

×