Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Living On A Cloud, Dr Keith Marlow

1,308 views

Published on

From presentation "Gaining leverate without the costs

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Living On A Cloud, Dr Keith Marlow

  1. 1. Living on a Cloud Dr Keith Marlow Chief Architect, APAC Region [email_address]
  2. 2. Agenda <ul><li>What cloud computing actually is… </li></ul><ul><li>The anatomy of a cloud.. </li></ul><ul><li>What does it really do for you? </li></ul><ul><li>Being cheap and leveraged </li></ul><ul><li>Yahoo API’s and Web Services </li></ul><ul><li>The future </li></ul><ul><li>Questions… </li></ul>
  3. 3. What Cloud Computing actually is… <ul><li>Really ancient history – way back in the early 1990’s… </li></ul><ul><li>All ‘in-house’ </li></ul><ul><li>All risks yours to manage </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage
  4. 4. What Cloud Computing actually is… <ul><li>Then in the mid to late 90’s – The Internet … </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage The Internet Not Business Critical! Full of none business ‘stuff’
  5. 5. What Cloud Computing actually is… <ul><li>Around 2000, the Internet became critical. </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage The Internet Full of businesses & customers! Critical to your Business!
  6. 6. What Cloud Computing actually is… <ul><li>But now with cloud computing we have.. </li></ul>YOUR BUSINESS YOUR DATA Control Processing Storage HOSTED SERVICES YOUR DATA? The Internet
  7. 7. What Cloud Computing actually is… <ul><li>Cloud Computing is either: </li></ul><ul><ul><li>Remotely hosted data processing services, or </li></ul></ul><ul><ul><li>Remotely hosted web services </li></ul></ul><ul><li>Which are: </li></ul><ul><ul><li>A highly distributed and flexible computing environment </li></ul></ul><ul><ul><li>With high availability </li></ul></ul><ul><li>Basically, it’s a remote self-scaling computing ‘resource’ </li></ul><ul><ul><li>Fire and forget </li></ul></ul><ul><li>Thanks, in part to: </li></ul><ul><ul><li>Cheaper bandwidth and hardware </li></ul></ul><ul><ul><li>Much faster machines </li></ul></ul><ul><ul><li>Abstraction and standards </li></ul></ul><ul><ul><li>Businesses, the research community & Open Source </li></ul></ul>
  8. 8. The anatomy of a cloud.. <ul><li>1 ST case – The Processing Cloud (Hadoop) </li></ul>MANAGEMENT SERVICES PROCESSING CLOUD 1000’s of nodes & disks DATA I/O CLIENT API ACCOUNTING & AUDITING
  9. 9. How does Hadoop scale? Map/Reduce Input Map Map Map Map Transient Data Results Reduce Reduce Reduce Reduce Split into ‘ bits’ Process the ‘bits’ on each node Collate each ‘bin’ on each node Shuffle into ‘ bins’ Join it all together
  10. 10. Hadoop – what is it? <ul><li>Open Source Apache project - http:// hadoop.apache.org /core/ </li></ul><ul><li>Hadoop Core includes: </li></ul><ul><ul><li>Distributed File System - distributes data between nodes </li></ul></ul><ul><ul><li>Map/Reduce - distributes application </li></ul></ul><ul><li>Written in Java </li></ul><ul><li>Runs on </li></ul><ul><ul><li>Linux, Mac OS/X, Windows, and Solaris </li></ul></ul><ul><ul><li>Commodity hardware </li></ul></ul>
  11. 11. Hadoop – How do we use it? <ul><li>Example: Web Search </li></ul><ul><ul><li>BIG graph: 100 billion nodes and 1 trillion edges </li></ul></ul><ul><ul><li>Largest shuffle is 450 TB (or 643000 CD’s worth!) </li></ul></ul><ul><ul><li>Final output is 300 TB compressed </li></ul></ul><ul><ul><li>Runs on 10,000 cores </li></ul></ul><ul><ul><li>Written in C++ </li></ul></ul>
  12. 12. Hadoop – How do we use it? What 20,0000 nodes look like
  13. 13. Hadoop – Real life usage… <ul><li>NY TIMES </li></ul><ul><li>Needed offline conversion of public domain articles from 1851-1922. </li></ul><ul><li>Used Hadoop to convert scanned images to PDF </li></ul><ul><li>Ran 100 Amazon EC2 instances for around 24 hours </li></ul><ul><li>4 TB of input </li></ul><ul><li>1.5 TB of output </li></ul>Published 1892, copyright New York Times
  14. 14. Hadoop – Who else uses it? <ul><li>Amazon/A9 </li></ul><ul><li>Facebook </li></ul><ul><li>Google </li></ul><ul><li>IBM </li></ul><ul><li>Joost </li></ul><ul><li>Last.fm </li></ul><ul><li>New York Times </li></ul><ul><li>PowerSet (now Microsoft) </li></ul><ul><li>Quantcast </li></ul><ul><li>Veoh </li></ul><ul><li>Yahoo! </li></ul><ul><li>Basically proven to be fit for purpose </li></ul><ul><li>More information at: </li></ul><ul><ul><li>http:// developer.yahoo.net/blogs/hadoop / </li></ul></ul>
  15. 15. The anatomy of a cloud.. <ul><li>2 nd case – The Web Services Cloud </li></ul>MANAGEMENT SERVICES DATA RETENTION PROCESSING CLOUD DATA I/O CLIENT API ACCOUNTING & AUDITING
  16. 16. The anatomy of a cloud.. <ul><li>2 nd case – The Web Services & Application Cloud </li></ul>MANAGEMENT SERVICES DATA RETENTION DATA I/O CLIENT API ACCOUNTING & AUDITING PROCESSING GRID 1000’s of nodes PROCESSING CLOUD 1000’s of nodes
  17. 17. Being cheap and leveraged… <ul><li>Cloud Computing allows time of usage outsourcing </li></ul><ul><ul><li>Only pay for exactly what you use </li></ul></ul><ul><ul><li>Lower CAPEX costs, greater ROI </li></ul></ul><ul><ul><li>Its green too! </li></ul></ul><ul><li>Needs a different approach to systems design </li></ul><ul><ul><li>New views on data to use in cloud & privacy protection </li></ul></ul><ul><ul><li>Decoupling around remote API’s </li></ul></ul><ul><ul><li>Remote hosting SDK’s training </li></ul></ul>
  18. 18. Being cheap and leveraged… <ul><li>Use the services of Yahoo! to implement and improve your services and offerings </li></ul><ul><ul><li>Low (or nil) set up and operational costs </li></ul></ul><ul><li>Yahoo provides the following: </li></ul><ul><ul><li>API’s and Web Services </li></ul></ul><ul><ul><li>RSS content feeds </li></ul></ul><ul><ul><li>Developer kits and GUI libraries </li></ul></ul><ul><ul><li>BrowserPlus </li></ul></ul><ul><ul><li>YAP </li></ul></ul><ul><li>To you they all operate as ‘clouds’ </li></ul>
  19. 19. Yahoo API’s and Web Services <ul><li>Maps </li></ul><ul><ul><li>Include a map on a website or intranet </li></ul></ul><ul><li>GeoPlanet ™ </li></ul><ul><ul><li>Geocode any address into latitude, longitude and WoeID </li></ul></ul><ul><li>Mail </li></ul><ul><ul><li>Send/Read email, lists folders etc </li></ul></ul><ul><ul><li>Zimbra – completely hosted mail service </li></ul></ul><ul><li>BOSS – Build you own Search Service </li></ul><ul><li>Search Monkey – Enriched search results for your sites </li></ul><ul><li>OpenID – share user ID’s between sites </li></ul><ul><li>YAP – Yahoo Application Platform – going Open on Yahoo! </li></ul><ul><li>http:// developer.yahoo.com / </li></ul>
  20. 20. BOSS <ul><li>API’s into Yahoo! Search </li></ul><ul><ul><li>Unlimited queries a day </li></ul></ul><ul><ul><li>No restrictions on presentation </li></ul></ul><ul><ul><li>Re-ordering allowed </li></ul></ul><ul><ul><li>Blending of Proprietary and Yahoo! Search Content Allowed </li></ul></ul><ul><ul><li>White-Label </li></ul></ul><ul><li>http://developer.yahoo.com/search/boss/ </li></ul><ul><li>Use to implement your own site search! </li></ul>
  21. 21. Search Monkey http://developer.yahoo.com/searchmonkey/
  22. 22. YAP – Yahoo! Application Platform <ul><li>Allows you to write applications modules that potentially can ‘run’ in any Yahoo! Web site (or supporting 3 rd party web site) </li></ul><ul><ul><li>i.e. Yahoo! itself becomes a ‘cloud’  </li></ul></ul><ul><ul><li>The user selects modules from a gallery to ‘paint’ onto their page canvas. </li></ul></ul><ul><ul><li>We take care of blending the modules with the content of the site to make the final page. </li></ul></ul><ul><li>What does this mean to you?? </li></ul><ul><ul><li>Ability to put your dynamic products and services onto Yahoo! </li></ul></ul><ul><ul><li>Closer relationship with users/customers in general </li></ul></ul>
  23. 23. The Future of Cloud Computing <ul><li>Smaller/Bigger/Faster/Cheaper </li></ul><ul><li>Hosting ‘in the cloud’ will become the norm </li></ul><ul><ul><li>Easier & cheaper than not doing so. </li></ul></ul><ul><li>Sum of the parts will be greater than the whole </li></ul><ul><ul><li>More ‘on the fly’ services aggregation & customization </li></ul></ul><ul><li>The user will ‘combine’ services to meet their needs. </li></ul><ul><li>The desktop PC/TV/Mobile will become the ‘presentation & personalization gateway’ into the Internet Cloud </li></ul>
  24. 24. Questions??

×