• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
397
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
5
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. 5 PILLARS OF CLOUDINESS wayne walls cloud evangelist alex brandt sr developer Tuesday, July 23, 13
  • 2. FIVE PILLARS OF CLOUDINESS 1 2 3 4 5 Parallel Modular Horizontal Agile Secure 2 Tuesday, July 23, 13
  • 3. Pillar #1: Parallel Tuesday, July 23, 13
  • 4. 1M Records Answer Node 4 Tuesday, July 23, 13
  • 5. 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M 5 Tuesday, July 23, 13
  • 6. 6 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Tuesday, July 23, 13
  • 7. 6 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Tuesday, July 23, 13
  • 8. 7 Tuesday, July 23, 13
  • 9. DATA CALCULATION MAP CALCULATION REDUCE DATA CALC D C R D D C R D D C R D SHUFFLE DATA 7 Tuesday, July 23, 13
  • 10. 8 Tuesday, July 23, 13
  • 11. Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 12. Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 13. File A Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 14. File A Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 15. File A File B Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 16. File A File B Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 17. File A File B Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 18. File A File B Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 19. File A File B File A Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 20. File A File B File A Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 21. File A File B File A Thread 1 Deadlock 9 X File B Thread 2 Tuesday, July 23, 13
  • 22. File A File B File A Thread 1 Deadlock 9 X File B Thread 2 Tuesday, July 23, 13
  • 23. File A File B File A Thread 1 Deadlock 9 X X File B Thread 2 Tuesday, July 23, 13
  • 24. Solution? Scheduling Y XA B A B Y X 1 2 3 4 10 Tuesday, July 23, 13
  • 25. 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 11 Tuesday, July 23, 13
  • 26. 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 11 Tuesday, July 23, 13
  • 27. 12 Tuesday, July 23, 13
  • 28. 13 Tuesday, July 23, 13
  • 29. pillar #2: modular Tuesday, July 23, 13
  • 30. ONE PERSON BUILDS A CAR 15 Tuesday, July 23, 13
  • 31. CHASSIS ENGINE BODY INTERIOR 16 Tuesday, July 23, 13
  • 32. CHASSIS ENGINE BODY INTERIOR 17 Tuesday, July 23, 13
  • 33. CHASSIS ENGINE BODY INTERIOR 18 Tuesday, July 23, 13
  • 34. CHASSIS ENGINE BODY INTERIOR SEATS FLOOR DASHBOARD GAS AND BRAKE PEDALS 18 Tuesday, July 23, 13
  • 35. CHASSIS ENGINE BODY INTERIOR 19 Tuesday, July 23, 13
  • 36. CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 37. defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 38. defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 39. defect defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 40. defect defect defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 41. defect defect defect defect LEMON CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 42. Pillar #3: horizontal Tuesday, July 23, 13
  • 43. 2 GB 2 CORES 22 8 GB 4 CORES Tuesday, July 23, 13
  • 44. 2 GB 2 CORES VERTICAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 45. 2 GB 2 CORES VERTICAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 46. 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 47. 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 48. 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 49. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 50. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 51. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 52. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 53. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 54. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 22 8 GB 4 CORES Tuesday, July 23, 13
  • 55. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 22 8 GB 4 CORES Tuesday, July 23, 13
  • 56. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 2 GB 2 CORES 22 8 GB 4 CORES Tuesday, July 23, 13
  • 57. 23 Tuesday, July 23, 13
  • 58. 24 Tuesday, July 23, 13
  • 59. ONE SERVER NEAR CAPACITY 24 Tuesday, July 23, 13
  • 60. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER 24 Tuesday, July 23, 13
  • 61. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER 25 Tuesday, July 23, 13
  • 62. 1234 MESSAGE QUEUE 26 Tuesday, July 23, 13
  • 63. 1234MESSAGE QUEUE 27 Tuesday, July 23, 13
  • 64. 1 2 3 4 28 MESSAGE QUEUE Tuesday, July 23, 13
  • 65. 1 2 3 4 28 56789101112 MESSAGE QUEUE Tuesday, July 23, 13
  • 66. 29 5678 9101112MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 67. 29 5678 9101112 ADD MORE SERVERS BASED ON MESSAGE QUEUE MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 68. MESSAGE QUEUE 30 5 6 7 8 9 10 11 12 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 69. 31 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 70. 32 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 71. 33 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 72. 33 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE DECREASE SERVERS BASED ON MESSAGE QUEUE Tuesday, July 23, 13
  • 73. MESSAGE QUEUE 34 5678 9101112 13 14 15 16 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 DECREASE SERVERS BASED ON MESSAGE QUEUE Tuesday, July 23, 13
  • 74. Pillar #4: agile Tuesday, July 23, 13
  • 75. 36 Tuesday, July 23, 13
  • 76. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 37 Agile - why Monday Tuesday, July 23, 13
  • 77. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 38 Tuesday Tuesday, July 23, 13
  • 78. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D 38 Tuesday Tuesday, July 23, 13
  • 79. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature G Agile - why 39 Wednesday Tuesday, July 23, 13
  • 80. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature D Feature F Feature G Agile - why 39 Wednesday Tuesday, July 23, 13
  • 81. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Agile - why 40 Friday Tuesday, July 23, 13
  • 82. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Feature H Feature I Agile - why 40 Friday Tuesday, July 23, 13
  • 83. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature EFeature D Feature F Feature G Feature H Feature I Agile - why 41 Friday - WK 2 Tuesday, July 23, 13
  • 84. Agile - why 57% 14% 29% Failed Successful Challenged 49% 42% 9% Waterfall Agile Source: The CHAOS Manifesto, The Standish Group, 2012 42 Tuesday, July 23, 13
  • 85. 43 Continuous Integration/Continuous Delivery Tuesday, July 23, 13
  • 86. 44 Quality Tuesday, July 23, 13
  • 87. 45 Release Cycle Tuesday, July 23, 13
  • 88. IMAGE: blogs.bu.edu 46 Tuesday, July 23, 13
  • 89. 47 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 90. ENSURES QUALITY 47 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 91. 48 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 92. ENSURES CONFIDENCE 48 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 93. Pillar #5: secure Tuesday, July 23, 13
  • 94. 50 Tuesday, July 23, 13
  • 95. 51 PRESENTATION APPLICATION STORAGE Tuesday, July 23, 13
  • 96. 51 PRESENTATION APPLICATION STORAGE INTERFACE SECURITY ACROSS ALL LAYERS Tuesday, July 23, 13
  • 97. 52 Tuesday, July 23, 13
  • 98. 53 Host-based Authentication Tuesday, July 23, 13
  • 99. 54 User Authentication Tuesday, July 23, 13
  • 100. 55 Hybrid Cloud Tuesday, July 23, 13
  • 101. 56 Trustworthiness as an Organization Tuesday, July 23, 13
  • 102. APPLICATION dissection alex brandt sr developer wayne walls cloud evangelist Tuesday, July 23, 13
  • 103. Data Schema •articles • url • text • tags-index • notations • location • note • votes • created_at-index • etag • parsed_at •users • username-unique index • email • name • hash (md5) 58 •subscribers • uuid4-user • uuid4-bookmark • subscribed_at Tuesday, July 23, 13
  • 104. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Tuesday, July 23, 13
  • 105. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 106. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 107. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 108. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 109. KEY/VALUE STORE OBJECT STORE LB FRONT END API MQ CONSUMER LB DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 110. 65 https://github.com/raxsavvy/margarine ThankYou! @rackspace @waynewalls @hhoover @alunduil Tuesday, July 23, 13