Your SlideShare is downloading. ×
0
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Unlocked Workshop OSCON 2013 - Part II
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Unlocked Workshop OSCON 2013 - Part II

435

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
435
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 5 PILLARS OF CLOUDINESS wayne walls cloud evangelist alex brandt sr developer Tuesday, July 23, 13
  • 2. FIVE PILLARS OF CLOUDINESS 1 2 3 4 5 Parallel Modular Horizontal Agile Secure 2 Tuesday, July 23, 13
  • 3. Pillar #1: Parallel Tuesday, July 23, 13
  • 4. 1M Records Answer Node 4 Tuesday, July 23, 13
  • 5. 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M 5 Tuesday, July 23, 13
  • 6. 6 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Tuesday, July 23, 13
  • 7. 6 1M Records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 751K-1M Tuesday, July 23, 13
  • 8. 7 Tuesday, July 23, 13
  • 9. DATA CALCULATION MAP CALCULATION REDUCE DATA CALC D C R D D C R D D C R D SHUFFLE DATA 7 Tuesday, July 23, 13
  • 10. 8 Tuesday, July 23, 13
  • 11. Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 12. Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 13. File A Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 14. File A Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 15. File A File B Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 16. File A File B Thread 1 Deadlock 9 Thread 2 Tuesday, July 23, 13
  • 17. File A File B Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 18. File A File B Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 19. File A File B File A Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 20. File A File B File A Thread 1 Deadlock 9 File B Thread 2 Tuesday, July 23, 13
  • 21. File A File B File A Thread 1 Deadlock 9 X File B Thread 2 Tuesday, July 23, 13
  • 22. File A File B File A Thread 1 Deadlock 9 X File B Thread 2 Tuesday, July 23, 13
  • 23. File A File B File A Thread 1 Deadlock 9 X X File B Thread 2 Tuesday, July 23, 13
  • 24. Solution? Scheduling Y XA B A B Y X 1 2 3 4 10 Tuesday, July 23, 13
  • 25. 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 11 Tuesday, July 23, 13
  • 26. 1M records Answer Node 1M records Answer Node 1-250K Node 251K-500K Node 501K-750K Node 750K-1M 11 Tuesday, July 23, 13
  • 27. 12 Tuesday, July 23, 13
  • 28. 13 Tuesday, July 23, 13
  • 29. pillar #2: modular Tuesday, July 23, 13
  • 30. ONE PERSON BUILDS A CAR 15 Tuesday, July 23, 13
  • 31. CHASSIS ENGINE BODY INTERIOR 16 Tuesday, July 23, 13
  • 32. CHASSIS ENGINE BODY INTERIOR 17 Tuesday, July 23, 13
  • 33. CHASSIS ENGINE BODY INTERIOR 18 Tuesday, July 23, 13
  • 34. CHASSIS ENGINE BODY INTERIOR SEATS FLOOR DASHBOARD GAS AND BRAKE PEDALS 18 Tuesday, July 23, 13
  • 35. CHASSIS ENGINE BODY INTERIOR 19 Tuesday, July 23, 13
  • 36. CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 37. defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 38. defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 39. defect defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 40. defect defect defect defect CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 41. defect defect defect defect LEMON CHASSIS ENGINE BODY INTERIOR 20 Tuesday, July 23, 13
  • 42. Pillar #3: horizontal Tuesday, July 23, 13
  • 43. 2 GB 2 CORES 22 8 GB 4 CORES Tuesday, July 23, 13
  • 44. 2 GB 2 CORES VERTICAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 45. 2 GB 2 CORES VERTICAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 46. 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 47. 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 48. 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 49. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 50. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 51. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 52. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 53. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL 22 8 GB 4 CORES Tuesday, July 23, 13
  • 54. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 22 8 GB 4 CORES Tuesday, July 23, 13
  • 55. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 22 8 GB 4 CORES Tuesday, July 23, 13
  • 56. 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES 2 GB 2 CORES VERTICAL HORIZONTAL SHARED NOTHING THROUGH STATELESSNESS 2 GB 2 CORES 22 8 GB 4 CORES Tuesday, July 23, 13
  • 57. 23 Tuesday, July 23, 13
  • 58. 24 Tuesday, July 23, 13
  • 59. ONE SERVER NEAR CAPACITY 24 Tuesday, July 23, 13
  • 60. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER 24 Tuesday, July 23, 13
  • 61. ONE SERVER NEAR CAPACITY ADD ANOTHER SERVER 25 Tuesday, July 23, 13
  • 62. 1234 MESSAGE QUEUE 26 Tuesday, July 23, 13
  • 63. 1234MESSAGE QUEUE 27 Tuesday, July 23, 13
  • 64. 1 2 3 4 28 MESSAGE QUEUE Tuesday, July 23, 13
  • 65. 1 2 3 4 28 56789101112 MESSAGE QUEUE Tuesday, July 23, 13
  • 66. 29 5678 9101112MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 67. 29 5678 9101112 ADD MORE SERVERS BASED ON MESSAGE QUEUE MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 68. MESSAGE QUEUE 30 5 6 7 8 9 10 11 12 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1 2 3 4 Tuesday, July 23, 13
  • 69. 31 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 70. 32 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 71. 33 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE Tuesday, July 23, 13
  • 72. 33 5678 9101112 13141516 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 MESSAGE QUEUE DECREASE SERVERS BASED ON MESSAGE QUEUE Tuesday, July 23, 13
  • 73. MESSAGE QUEUE 34 5678 9101112 13 14 15 16 ADD MORE SERVERS BASED ON MESSAGE QUEUE 1234 DECREASE SERVERS BASED ON MESSAGE QUEUE Tuesday, July 23, 13
  • 74. Pillar #4: agile Tuesday, July 23, 13
  • 75. 36 Tuesday, July 23, 13
  • 76. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 37 Agile - why Monday Tuesday, July 23, 13
  • 77. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C 38 Tuesday Tuesday, July 23, 13
  • 78. Agile - why Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D 38 Tuesday Tuesday, July 23, 13
  • 79. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature G Agile - why 39 Wednesday Tuesday, July 23, 13
  • 80. Features In Progress Blocked Tabled Testing Complete Feature AFeature BFeature C Feature E Feature D Feature F Feature G Agile - why 39 Wednesday Tuesday, July 23, 13
  • 81. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Agile - why 40 Friday Tuesday, July 23, 13
  • 82. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature E Feature D Feature F Feature G Feature H Feature I Agile - why 40 Friday Tuesday, July 23, 13
  • 83. Features In Progress Blocked Tabled Testing Complete Feature A Feature B Feature C Feature EFeature D Feature F Feature G Feature H Feature I Agile - why 41 Friday - WK 2 Tuesday, July 23, 13
  • 84. Agile - why 57% 14% 29% Failed Successful Challenged 49% 42% 9% Waterfall Agile Source: The CHAOS Manifesto, The Standish Group, 2012 42 Tuesday, July 23, 13
  • 85. 43 Continuous Integration/Continuous Delivery Tuesday, July 23, 13
  • 86. 44 Quality Tuesday, July 23, 13
  • 87. 45 Release Cycle Tuesday, July 23, 13
  • 88. IMAGE: blogs.bu.edu 46 Tuesday, July 23, 13
  • 89. 47 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 90. ENSURES QUALITY 47 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 91. 48 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 92. ENSURES CONFIDENCE 48 Lint Testing Test Driven Development Functional Testing Black Box Integration Testing Unit Testing Behavior Driven Development White Box Smoke Testing Acceptance Testing Tuesday, July 23, 13
  • 93. Pillar #5: secure Tuesday, July 23, 13
  • 94. 50 Tuesday, July 23, 13
  • 95. 51 PRESENTATION APPLICATION STORAGE Tuesday, July 23, 13
  • 96. 51 PRESENTATION APPLICATION STORAGE INTERFACE SECURITY ACROSS ALL LAYERS Tuesday, July 23, 13
  • 97. 52 Tuesday, July 23, 13
  • 98. 53 Host-based Authentication Tuesday, July 23, 13
  • 99. 54 User Authentication Tuesday, July 23, 13
  • 100. 55 Hybrid Cloud Tuesday, July 23, 13
  • 101. 56 Trustworthiness as an Organization Tuesday, July 23, 13
  • 102. APPLICATION dissection alex brandt sr developer wayne walls cloud evangelist Tuesday, July 23, 13
  • 103. Data Schema •articles • url • text • tags-index • notations • location • note • votes • created_at-index • etag • parsed_at •users • username-unique index • email • name • hash (md5) 58 •subscribers • uuid4-user • uuid4-bookmark • subscribed_at Tuesday, July 23, 13
  • 104. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Tuesday, July 23, 13
  • 105. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 106. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 107. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 108. LB FRONT END API MQ CONSUMER LB KEY/VALUE STORE DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize OBJECT STORE Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 109. KEY/VALUE STORE OBJECT STORE LB FRONT END API MQ CONSUMER LB DOCUMENT STORE INTERNET Routing •Topic •users •articles •Fanout •article.sanitize •article.references Consumers •create user •password user •article reference •email user •create article •article sanitize Parallel Modular Horizontally scalable Agile Secure Tuesday, July 23, 13
  • 110. 65 https://github.com/raxsavvy/margarine ThankYou! @rackspace @waynewalls @hhoover @alunduil Tuesday, July 23, 13

×